So, there was some talk about removing fire systems from the kernel, and also, this unprivileged security issues with mounting fire systems by user namespaces and that sort of thing. And my idea would be that we run a kernel in a VM just for mounting a fire system. And so, and now that, it fuse. And so, we could run all the kernels, which still support all of the fire systems we want to remove. And I see what you're saying, so you, like, pass through from the VM as a fuse mount for any old file systems that we remove that for users that might still have it. Christian, are you in here? Oh, yeah. Okay, good. I just want you to hear what I'm going to say. I think that we are extremely conservative when it comes to removing file systems when we shouldn't be. I definitely agree with Linus's point of view that we shouldn't be breaking user space, but like the NTFS thing is a really good example of this, is like, if it's been in tree for like three months and people just disappeared, delete it. Like if somebody wants it, they come back and they do the whole thing and then they promise that they're actually going to stick around. I think RyzerFS is a different beast, but like a lot of these other file systems, like I think we should be extremely, extremely aggressive about deleting things we just don't care about anymore because if there's one or two users, great, that's what stable kernels are for. They can, or they can buy RHEL, or they can do literally anything else, but for the upstream kernel, if there's no maintainer and nobody's using it, let's delete it. I don't understand why we have to keep all this code around. But it still could be some file system image lying around that somebody wants to mount. But you can't use it because GCC has gone, so you can't compile it anymore. But if we still have a certain kernel which we maintain with all these file systems in it that still compiles, at least the file system still compiles with the recent GCC, then that would, I guess, solve that issue. But not sure, that's still some maintenance. I wonder, so I think the only way we can, so with the file systems that we just mounted and it haven't been upstream for long, I think that would make sense. The only thing that we would need, we would basically need to ask Linus to maintain our summit or whatever and be like, hey, how would you feel about this if we were to switch to this type of policy? But in general, I think it makes sense because it's really, it's a burden, I think, on the mailing list, a burden on the other maintainers if just patches and Sysport reports keep accumulating for file systems that have been there for a very short time. And using a VM, UML might make a better option. UML, user mode linux. And I'm not sure UML is still working. In the older kernels you're talking about, it might be, it's a thought. The other thing is, so mounting files and images, arbitrary images without any checking, the user wants to mount VFET or anything, that's another use case for this. So I think that if someone was willing to implement some sort of run UML in a VM or some other sort of kernel in a VM, whatever, to ease the fear that there might be potential user pain, it may make it easier for linux to agree to remove a file system with extreme prejudice. There may be disagreement in this room of people of good will about whether or not that is strictly speaking necessary, and we could have that debate. I do think that UML may be an interesting path forward because UML works well enough for K-Unit, so I suspect UML hasn't completely bit-rotted at least for X86, because that's what K-Unit uses, and that might be an interesting way of, we somehow have to feed the block IO requests in through the VM into UML or whatever kernel we use, and then feed it back out through some sort of vert IO into the fuse on the host kernel. So there's real technical work there. I think it'd be really, really cool if it was done just so that we don't have to have that discussion about is it realistic that people be able to build an LTS kernel from two years ago, if that's the only way they can get a particular file system, because the reality is I can't build, I think it's 4.15 or 4.14, there are older kernels that are not buildable with GCC anymore, and so yeah, that's a concern. Whether or not that should stop us from removing a file system, and I agree that RyzerFS is probably a little bit different from NTFS-3 about what the decision matrix should be, but like if you think you can actually do the fuse thing, I think that'd be really cool, right, that just from a technology perspective, and it just short circuits that entire argument. Yeah, so that's the... Just for a second, you've mentioned UML being not entirely dethrone on X86, just to check it never existed outside of X86. The last time I heard this argument come up was several years ago when we were here in Vancouver for the Plumbers Conference, and we were talking about accessing files that we would... Our file systems we had find in USB drives we picked up off the street, and the solution offered then was why not start up a VM and export the unknown file system via a shared network file system like SMB or NFS. You don't have to deal with fuse or write any new code, you just mount it in that file system and export it. Unfortunately, USB drive picked on the street might do rather unpleasant things to you simply when you stick it in without any... So kind of the other thing that I'm getting at here, right, is we use the specter of unknown user to avoid making this decision all of the time. And I'm quite tired of using arbitrary things... Arbitrary fears with no data backing it. So I think Ryzer FS, again, is a clear exception because there are real users, there are enterprise users that are still on Ryzer FS. Great, we don't eat that guy. But we can easily go look at SLEZ or OpenSUSA or Fedora configs and see what file systems are actually turned on and delete everything that's not turned on because chances are there are no users. Of course, if something the user complains, then we need to have a conversation, right, but like we constantly use the specter of some unknown user.  Okay, we can't build 4.14, but we can build 6.2. And so we eat something right now and we find out that there's a user. Then we can decide, okay, do we care about that user? Because they have 6.2 and literally many other options. And now I do agree with you, Ted, that this is a cool technology of being able to export things and solve this problem in general. But at the same time, I am so tired of being like, well, there might be a user, yeah, okay, there's lots of other options. I mean, this is a slightly different scenario, of course. But we do it with architectures, right? At least from regular intervals, we keep, let's remove this architecture. And then users will pop up and they will tell us, no, you can't. But that's how we got rid of a bunch of architectures over the years. And it's kind of similar, right? So, you know, when you start to get into the CPU, you stop getting produced, but still there is some university who is still like two machines in their basement that they whip out for a Christmas party. Or so they can use an old kernel. There is one slight difference you need to consider, which is whether the file system is usable as a root device. So if it's not like NTFS, I don't think anybody will care. You just want access to your data. But the problem with root device file systems is you can't upgrade once the file system is gone.  I mean, yes, but that's again, that's, I think, the argument of the invisible user, the hidden user, like who's going to run a riser-FS file system and then upgrade to a butter-FS root file system. Yeah, so actually, I was deprecating the riser-FS stuff about an year ago. And so I've got a couple of users complaining, hey, I'm still using it. But basically, all of two of them were kind of fine with the statement. Okay, but so we will remove it in two years. And do you know that you are running a code base that has so many security bugs that you probably don't want to think about it and say, okay, yeah. I thought that when it is in kernel, it must be still fine and maintained. So, okay, thanks for the notice. I am going to switch to something else. So like all these users, when they are given sufficiently advanced notice, they are willing to transition their machines to something else. So I believe it is really the path forward to kind of make light noise, this code base is really bad, just use something else and give them like better suggestions that they can use. So with riser-FS, I'm still kind of confident that we can follow the precation notice and remove it like in one year. Or I don't know how long we set it, but about in one year, I believe we should be able to remove it. I think ultimately though, it would be useful for us to get a clear statement from Linus. So I think that probably is the next step. Whether we do it via email or we say, this is something we want to queue up for the maintainer summit. Because I was against NTFS3 going in, it wasn't up to me. And it's like, okay, it's in, I'll step back and have some popcorn and watch. But like, so I have absolutely no objections if NTFS3 were ejected tomorrow. If it would make it easier for Linus to agree to remove it, by having this cool technology, I'm all for it. Because I never wanted NTFS3 in the kernel in the first place. But again, it's not up to me, right? Ultimately, I think we need to escalate this to Linus. And if we're willing to have a sense of the file system developers in the room say, we should be a little bit more aggressive about being willing to remove deprecated file systems like we remove deprecated architectures. I'll certainly sign on the dotted line for that. Yeah, I think this is what I'm- Go ahead, Derek. So I was talking with Kent about an hour ago about this. And I was thinking, if there was some way for me to pass IOMAP mappings through views or something like that to IOMAP, I was thinking, why can't we just get rid of XFS from the kernel?  Yeah, let's put things in there and kick them out in six months if they're not making progress and generating too many sysbot reports. I have been tried. It was in driver stage and it was looser. And it stuck around. I'll skip the adjectives I would like to apply to it. But put it that way. It's something that sticks to the boot so. And it stuck there for quite a while. It had been hard to out of the kernel. Right, and I think this is the thing is like, as a community, we've all been kind of willing to accept that there's like, okay, whatever, it's not that big of a deal. But we're finding out that it is a big deal. And I think that as a community, file system maintainers, VFS maintainers, I'll go to Linus and say, listen, we would live a lot more aggressive about removing stuff. I think Linus would be like, okay, that's reasonable. And I think that we just need to be unified in this idea that like, okay, there is real cost to carrying the shit around. Let's be a lot more aggressive about evicting it when it's clearly getting in our way. Because it makes us look bad. Yeah, it makes us look bad. First one on staging, if you have a file system staging, you will have staging for entry to file systems as well and you've lost control of what file systems go in. And the second one to Ted's point is, I really don't think you want to persuade Linus to make a statement. I think you want to take a statement to Linus and tell him this is it. That's exactly what I'm saying is like, I don't think Linus cares too, I mean, he does. But like, I think he's going to, in this particular case, he would defer to all of us. And if all of us come up to him and say, we want to get rid of stuff as quickly as possible and be a lot more aggressive about when we bring in things and there's clear failure of maintainership, NTFS-3. Or, you know, there's clearly no users or there's just, if there are users, we just simply don't care because it's too old, it's unmaintained and it's causing us problems. Let's get rid of it. And I think that Linus would be okay with that. Eric's okay with it.  I mean, happens all the time. I think it would make sense. Chuck just asked for a short list. I think JFS, ROMFS, RAMFS maybe, HFS. RAMFS, you're kidding me. Not RAMFS, the compressed, CRAMFS, sorry. Fucking, what else, God, now I'm blind, I'm blind. You know, 65 things concerning. Yeah, there's a bunch.  Like, ButterFS, EXT4, XFS, NFS, SIFS, ZoneFS, F2FS. And you're looking for something. No, I just saw, yeah, it's maintain. There are strange folks who like AFS. AFS, we actually did get rid of ZXFS. The Veritas one, yeah, the Veritas one, sweet.  How much maintenance do you need for MinixFS, for example? How much maintenance does it take to keep, say, MinixFS alive? I have no idea. I think the price is usually paid by... It's not a great burden from what I can tell. Until the, you know, SysBot report comes along and you really have to, yeah, probably also for you, it might be different, I think, because you're so experienced that for you, it might be obvious to maintain them. But the thing is, if we have SysBot reports for these file systems coming in and then fixing these SysBot reports and bugs, who is actually doing it apart from maybe you and if I know what's going on sometimes myself? And they just accumulate if you look into the SysBot instance and, you know. I don't know, in my experience, SysBot reports often is not random garbage.  New-mounted API, IOMAP. Like, there are big changes that we are halfway through and likely won't have somebody to do for all these other things. So we're just also not, we're not only carrying around these dead file systems, we're also carrying around all of this dead code and all these dead interfaces that new developers are gonna come upon and be like, I don't know what I'm supposed to use. And so like, there's a real value in cleaning these up because they can also then clean up all these dead interfaces that we don't need anymore. I was gonna refer back to something earlier, you know, maybe SysBot is the answer. If the SysBot reports are piling up for a file system and, you know, there's no particular motion on many of them, that maybe that's how we know it's time to get rid of that particular file system. I was wondering, how many of the buffer head file systems can we get rid of just to simplify the buffer head? If the answer is 23, we can just get rid of buffer heads. I suspect we can't get rid of the ISO 9.660, maybe. But if that's the only one left. James says it's an archiving problem. And that the archives get annoyed because they store things on DVDs that they were promised will rest, the data there will rest for years. You get to it 50 years later and they can't find anybody who will read it. So if ISO 9.990 still works and it's not a burden, just leave it, because somebody will complain by the time all of us are dead and there'll be no one to fix it. Sounds like the laser disk problem that they had in there. Yeah, but okay. Yeah, I'll also note though that for a number of these file systems, in particular ISO 9.660, that's a classic, why isn't there a FuseFS for the damn thing, right? Because we don't care about performance for ISO 9.660. I agree. So Ted will write the FuseFS and you can remove it. Well, yeah, but yeah.  It's ISO 9.660, it's an archive format. Why is it not simply a TAR format, right? If you use TAR to extract ISO 9.660 to your current file system, isn't that perfect? Yeah, so there is a Fuse ISO 9.660 already. So I don't know how well it works, but like it exists. Does it work for root? It doesn't mean it works. So yeah. It exists, good enough. Does it work as a root file system? Yeah. Fuse makes it up though. Yeah, the other thing maybe to consider here also is it's not just Syspot reports. There are a number of these file systems that actually still have XFS test support. And every once in a while, I'll run XFS tests on some of these ancient file systems just for grins. And it's like, gee, that's nice. There's over like 150 different failures. I ran it on RyzerFS as part of my, because I had this insane idea that for every file system that is runnable via XFS tests, I bucked a mirror about this on WhatsApp a lot. I ran it and it was just like complete mess, like kernel crashes, everything failed. You didn't know what from run to run different failures. It's really not great. Yeah, the last time I tried running UDF with XFS tests, the kernel crashed. So yeah, I know. Look, the thing is here, if you're doing VFS change, or a folio changes, for example, and you're making a change to a file system that you cannot test, then it's kind of wrong. It's kind of better to remove it. But yeah, it's not very practical, but that's basically the correct answer for development. It's not good to change code that you cannot test, but. Is something like ISO 966, these read-only file systems, or are they something that's easier to convert to IOMAP because you don't have to worry about all the write back stuff, all the synchronization stuff? Can you just, instead of using BV, just came out like a buff and read into it? Actually, what makes read-only file systems particularly good from an IOMAP conversion point of view is you don't have to understand what get block with the create flag set to true does. You just have to understand what get block does in terms of getting a block rather than actually trying to allocate new blocks, which is what I've always found hardest about trying to convert something to IOMAP. I think the biggest problem is something like EXT2. And thank you, Anne, for volunteering to take that on, but where you actually do still want to preserve the read-write nature of it. Yeah, the one thing I was gonna say about EXT2 is I think EXT2 being well maintained as a simple file system is one of the best arguments for why we don't need to keep Minix FS, right? The original reason why we would point at Minix FS was, okay, you're writing a simple file system, Minix FS is what you should look at. I think these days, if we want a simple file system that people should look at, we should point them at EXT2, and that's one of the reasons why I'm really glad that it's going to IOFS, because it's like, it's a well-maintained simple file system using the modern interfaces, right? Now, Minix FS may have sentimental value because it was the first file system in Linux, blah, blah, blah, that's not a technical discussion, and I don't care. We have good chemistry. We can't even run Minix binaries anymore, I think, because we took ADOT out, so that's a sentimental attachment, but yeah, we can talk to Linus about it, but I would be all in favor of taking out Minix. It probably isn't that bad, but I don't think it has a reason to exist anymore, right? And this is sort of the, like, there are real booting faults. Second, Al. Al, can you repeat that? There are faults who are dual-booting Minix and Linux, strangely enough, because there had been, for real, patches supporting the new version of file system, Minix 3, that went into the tree, doesn't seem to be particularly obnoxious, so somebody cares about that. Had been a few, some years ago, I don't remember when, but strangely enough, it's not embedded. Yeah, unfortunately, there is actually an active Minix 3 community. I think it's pretty small, and I didn't even realize that they were actually interested in doing Linux file exchange, but whatever. I mean, it doesn't have to be, like, Minix 3 is maybe just a, we don't need to get hooked up about this specific file system. If we find out that we need to keep it, then that's perfectly fine. I mean, this really isn't a story about ripping out people, like the file system that a lot of people had. It's a story about trying to get rid of legacy code, and if anybody complains and has valid complaints, then we'll happily revert it or leave it in. That's at least how I would see it. Can we actually reduce some of the read-write file systems for really old legacy formats to just read-only file systems and just simplify them a lot? I mean, I would be, you know, if that's like an intermediate option, I mean, I'd prefer to just remove, obviously, but like, I think as long as we, I don't know, see, like, this is the problem, right? It's like, there's still things like the amount API changes, and IonMapOK is easier in this case, but what next API change are we gonna have that suddenly is a pain? And additionally, if we have no way to test it, oh, I know. But additionally, if we simply don't have any way to test it, and we're all around here running around updating APIs, and we just simply break the file system because, like, we can't test it, like, that's not great either, so. I mean, I'll note for architectures, that actually has been a feature where we accidentally break an architecture, and then we point out, we broke this architecture three years ago, no one stepped forward to complain, and then that makes it a whole lot easier to remove the architecture. And also, do we remove things that we don't have a MacFS in the F6.4? Oh, I wouldn't. That, I feel. For example, MkFS for UFS is quite available, not packaged. Not packaged. Of course, these defaults can't. It doesn't work for things like ISO file systems because they have CD record, which does exactly, namely creating an ISO without being in MkFS. Yeah, I think that's sort of a special case, and we're some of them like, I have no idea whether or not there are anyone who's using the QNX6 file system for interchange with QNX or BFS, right? I'm pretty sure ADFS probably doesn't have a whole lot of users, because like OSF-1 is kind of dead, dead, dead. But, you know. While at it, look at the partition schemes, please.  Yes. Okay, I think we're all in agreement here, and we're all just kind of going around in circles, so let's wrap this one up. Does anybody else have any other lightning talks? I know Chuck, you're still waiting. Do you want, are you good with who's in the room now? Yeah, okay, cool.  This talk is for Christian and Al and Andrew Morton. Where did Christian go? Nope. You moved. The issue is tempFS, stable directory cookies. People have heard me preach about this before. On the mailing list, the issue is that when you delete or create a file in a tempFS directory, the offsets change. And there's a sort of a simple cursor scheme that allows applications to find their way around with getDents, at least when they're accessing tempFS directories locally, but over NFS, that doesn't work because every reader on the NFS server opens the file and then closes it, which throws away the cursor. My proposed solution was to replace the cursor mechanism with something that is an XRA-based mechanism for finding dentries, based on their offset, and then every file gets an offset that is the same for as long as the directory lives and for tempFS, that's as long as the system is booted. There's been some grumbling about doing this. It looks maybe more complex than people have stomach for, so I'm looking for feedback about what might be an alternate approach that is simpler and performs well. Another suggestion was maybe we should, instead of changing tempFS, we should actually plumb all of this into libFS, which is why I'm kind of looking at Christian. We can put it into libFS, but... Go ahead. We can put it into libFS, but there is no real difference because tempFS is mostly a use of libFS functions with the distinctions. It uses a lot of the simple calls, but there are certain ones that tempFS has its own rename and that could get a little sticky, but tempFS uses the simple fs, reader, and lseq. Yep. Is there another comment? You want something like xr8 to directory reference to what the interest of children? Yes, that's exactly what I did. Yeah, and it works well. There are some performance regressions I have to look into, but I seem to have addressed all of the functional problems with it, maybe not all of the political problems with it. Sounds fine to me and it makes sense to put it into libFS, I think, even though I'm not super experienced in this area, but what did the tempFS maintainers say? Which is you probably, right? Hugh didn't comment on it. Andrew suggested it was a little complicated looking and it is file system specific code. Maybe it's a little outside the wheelhouse of the MM folks and maybe they would rather see it done in libFS as well just because it becomes not their problem. Okay, just a sec. So, okay, I have probably not looked your patches here, would you remind me of the subject? Subject line you used, one question of them, if they fit the question. Yeah, I don't remember the exact subject line, but I can send it to you once we're done talking here. Okay. In principle, I don't see any great problems. I don't know if it's worth actually, but the same logics for regular friends is used for, unfortunately, for some synthetic file systems, if I'm not correct. Yes. TempFS doesn't have to deal with random changes of directory contents coming out of hell knows what source was, it's all from user. So, I would need to actually take a look at horrors like, what was that, libFS? Some, there are completely insane specialist file systems, something in a band. I'm sorry, which are using that. Figuring out when and how they change the director is, is not something I would wish upon anyone. Right, as dev. So, maybe it's better to, I have no problem with that being in libFS, just not replacing the existing variant. One idea there was to actually add a different version of reDur and Durl seek just for the X-ray version and then have the, the weirder file systems continue to use the simple one that's already in there. How do you get from I0 to X-ray? The X-rays. File system specific. Yes, it is. So, how would those library help us deal with that? I'm not sure. What we have there right now can be directly used as instances of get the answer. Okay, iterate and lc. If it becomes a helper function that should be called by that file system's instance of file seek for directors, given it's some kind of callback or something like that, that's a bit of additional headache. It's really, the situation there was in the details. I would need to take a good look at your page settings see if, how widely can that be used without excessive headache. Yeah, fair enough. My patch that is currently Shemem FS specific. I haven't actually done anything with libFS itself but we could think about that offline. Thank you. I obviously have no problem with that. All right, great. Do you guys have any other lightning topics they'd like to discuss? Excellent. Oh, really. I tried to sneak this into the MM track but Michael gave the slot to Andrew instead. I want to reduce the amount of support we have for Hymem in the kernel. Like when Hymem was originally introduced, we were talking about like x86 servers with eight gig or theoretically up to 64 gig of memory but I don't think that ever worked. So we're talking about like 32 bit x86 systems. These days, I don't think anybody sells a 32 bit x86 system anymore and the 32 bit x86 kernel is almost unmaintained. So really the only 32 bit systems we actually care about I think are ARM based and even then it's really hard to buy a 32 bit ARM system with an amount of memory that causes you to actually need to use Hymem. Intel does not still sell Quark. Quark was an experiment and it failed. I have one. I've never used it. Just for the record, Fedora just recently disabled 32 bit ARM altogether and Fedora 38. Yeah, I mean for a while you've been able to use a 32 bit user space with a 64 bit kernel and I'm not proposing doing anything about that. I'm really talking about systems that can only run a 32 bit kernel. I don't propose entirely killing Hymem because I mean if you have more than I think about 800 megabytes of memory, you do still need it and you can buy like a ARM router or something with a gigabyte of RAM and it would be nice to still use the page cache and still use anonymous memory. So I'm talking about keeping those but I do want to start removing support for things like page tables in Hymem because that actually gives us a bit of software simplification. I want to remove keeping directories in Hymem. I want to other file system metadata.   One knows curiosity question. So a lot of the Hymem architectures also support a four gig, four gig split. It's just supposed to be slower. I think all of them do in which case can we not just disable Hymem as a performance optimization and keep the four gig, four gig split? I don't think ARM supports four gig, four gig split. I don't know. I mean, as I recall, four gig, four gig split, next city six was incredibly slow. Like, I mean, not just a little bit slow but like, but benchmarks ran half as fast. Like, it was, it was really, really bad. Well, yes, it's those two, but I mean people, people do still buy routers with like a gig of RAM, right? And less than a gig of RAM. So, am I hearing any objections to starting this work? Now, like, ButterFS, we went through and like ripped out the Hymem support for our metadata. So like, I don't think you're gonna find too much argument here, because like, you know, for us, it drastically simplified things. And I think that, you know, if anybody else still does it, I don't know if anybody, like, I don't think you're gonna find a lot of resistance to it, right, because it like, it sucks to do all the K-map stuff, right? Yeah, yeah. I mean, specifically for folios, right? Because you can't K-map an entire folio. And it just makes everything more complicated. And it's like, why, why are we straining so hard at something that matters to so, so few people? Yeah, I mean, I know we did it because it just was silly, right? And so like, I don't think you're gonna find a lot of people that are gonna complain loudly about it. So I'm all in favor of it. I'm seeing a lot of nodding. Okay, thank you. All right, any last minute lightning talks, perfect. All right, this one a lot longer than I expected. We're gonna go out here and we're gonna take a group photo. And then we have dinner at 6.30, and it's, I forgot the name of the restaurant. Steamworks, thank you. It's right down the road over here. It's on the back of your badge too. It's in the, there's a Google Maps link in the schedule. I cannot stress enough, 6.30. So find something else to do for a while.  And it's really close, so you can walk through it. But first, out here for the photo. Thanks everybody.


The All Progressive's Congress APC in Oyo State has described Saturday's killings in Ngongong, Ibarakpan North, a local government area, as one to many. An urged Governor Xi Makin Day to promptly convoke a security summit. The APC said the summit would seek to find a lasting solution to the security challenges facing the state. This was contained in a statement signed by its publicity secretary, Dr Abdulaziz and made available to newsmen. Recall that several people were killed in Ibarakpan area of Oyo State on Sunday night when a non-government invaded the community. The APC appealed to Makin Day to forget the rhetoric of having Amoteco and Operation Burst and that both had shown to be ineffective. Find about fresh news updates.


Stevenson falls to Gettysburg tonight. 10-5, Glenn Clark, MC McFadden with Coach Paul Cantabena. Coach, I know there's a few issues that we'll address during this, but one that jumps out at me, you faced a goalie who made 18 saves. How much of that was on him? How much of that was on your shot selection? Well, I think we took some shots that probably went the best, but we had some really quality shots, especially early on and in the third quarter. And he made the saves. 18, we only had seven. And it's tough to lose to a team that only takes 24 shots all game. So we got to do a little bit better job there. But I thought they were just a little more tougher than us. And I thought they wanted a little bit more. And I thought our ground ball play was not well. So we got to do a lot better at that aspect. But we got to play harder. I just don't think we played as hard as we did the first couple of games, for sure. It started out well. It started out with the 2-0 lead. One issue was definitely your extra man offense tonight. Yeah. I mean, discarded off 2-0 at the beginning of the game. It's a long game. So it doesn't really matter how you start off. It's how you finish. And so we got to do a much better job on the man up the situation. And we had plenty of opportunities to do that. But we're a good dad a week ago. And I saw our guys are feeling a little too good about themselves. We win one game. That's what our problem got to get better and better as we go along. And they got to show a little more fire, that's for sure. Especially our younger kids. I thought we played like freshmen today. It was just not a day that we didn't just were just a little disorganized all together. And we got to coach them better, that's for sure. We saw some different things at the face off dot in the fourth quarter. Put a couple guys out there. Give them some burst looks. What's the strategy moving forward? Well, I think Shiloh will be fine. I think he can do better. Our wing play was not good today. I thought they kind of bullied us. The two poles are pretty good. They do a really good job of that. And I didn't think we competed very well until we put two poles out there and kind of combat them a little bit. Shiloh will be just fine. We just had to give them a different look. And it worked for a little while for there. But it wasn't really Shiloh's fault.  And I think we got something we'll work on a lot today this week. We were curious about the flag that was thrown when the game ended. I have no idea what it was. All right. The other thing that sort of jumped out tonight, neither Ethan nor Jake was really able to get going, necessarily. They both ended up coming away with a goal there at the end of the game. Was it something they were doing defensively? I think Jacob's a little banged up. So he had a couple of games.  And he's pretty sore. But he's fighting through it. And I think he'll be much better next week. Give him a couple of days rest. And I think he'll be much better next week. And Ethan 35 is a really good defenseman. He's a second team All-American a year ago. It's probably a first team this year. And Ethan thought, you know, he gave, we went at him. And, you know, Ethan had some shots that didn't go. And he got one late.  But I don't think Ethan was, that was the issue.  And we had good shots. And I thought we had a lot of really good shots. And they just didn't go. And the kid ended up with 18 saves to our 7. And we held down the 24 shots. You think you'd win a game that way. And we should have won a game that way. I just didn't think that we competed the way we needed to do. Well, it gets no easier, obviously. You head on the road to go take on Christopher Newport this week. And that's why you schedule it like this, right? Yeah, and we'll learn. We're learning how to do things. And obviously, we learned how to not to play today. But, you know, you learn everything from these things. We'll take a lot of some positives and take some negatives and things we need to work at. But, you know, we've got to do a few things better in order to compete day in and day out. You know, we played two really good teams the first two weeks, two top 10 teams. And we competed really well. And then, you know, Gettysburg is another. I think they're going to be just fine there. Top 10 team as well. I think in the end, they'll be back up there. And, you know, we play a good schedule. Then we play Christopher Newport. And, you know, we'll get ready for them. And then get ready for Frank Little-Marshall. And then we'll get ready for Denison. And then we'll get ready for Lynchburg. And then we'll get ready for Salisbury. But, you know, we're trying to get ready for the conferencing. And we'll be fine. We'll win some of those games. And we're competing. And I think our guys care. They're invested. And it just didn't go our way today. Coach, appreciate it. Thank you very much. He's Paul Cantabena. And this is Go Mustang Sports dot com.


No doubt, achieving financial freedom and living in our terms is what we visualize when we think about our future. Still, many people will never achieve this dream of being financially successful. The reason why some individuals will stay broke is not that they cannot be economically independent but because they have a wrong mindset and lousy attitude towards finance. In this video, I will share with you 9 common reasons why people stay broke. 1. Lack of financial goals If you don't want to be broke, one of the foremost things you need to do is to set financial goals. Anna Maria, Lucardi, a George Washington University professor and expert in debt management once said that the bottom line is everyone should do more to plan for their financial future. Setting financial goals is like saving your present and securing your future. It helps to coordinate your activities such as helping multiple income streams, expenditure, investments, getting off debt, etc. and also future events such as retirement, insurance and having an excellent financial house upon your death. When you do not have plans for your finance, you are bent to remain poor. 2. Spending more than they earn When your expenses are more than your means, then you might never break free from being broke. Ideally, your costs should not even equal your earning.   Do you know that it has been modified to cut your clothes according to your material? You may wonder what the difference is between the two. Well, cutting your coat according to your size means spending according to your wants. However, it's better to cut your coat according to your material because your capacity may be bigger than your material and you need to reserve some material for later use. Many people make expenses based on their wants and that amounts to them staying broke, even though they have a job. 3. Kipping up with the Joneses A lot of things that people, especially the poor, spend their money on are things that they do not need.  This need to keep up with peers and get closer to those you perceive as superior is termed the Jones effect. While this need can be a driving force for brands, it harms the finance of individuals with a poor mindset. If you have the dream of leaving a bar being broke, then you must listen to this piece of advice from American radio host Dave Ramsey. You need to live as no one else would now, so in future you can leave as no one else can. Your biggest competition is not your next door neighbor who just acquired a new car, but yourself who finds it hard to save and invest because you want to compete with others. 4. Not Investing Investment is a medium through which you can build real wealth. However, many people give the excuse that investing is like gambling and is complicated. They settle for the belief that if their income increases, their life will be better. But no, that belief is fallacious. An increased income will not suffice for your being broke. You need to make your money multiply itself. Although investment is a risk, it is one that is worth taking because of its benefits in the long run. While broke people struggle to pay interest, the wealthy earn interest from the money they consistently put into investments. 5. Making Impulsive Purchases Have you ever been in a position where you had to wonder where all your money went? Well, here it is. The last time you went to the grocery store, you spent above what you budgeted on something that was not necessary. A new survey revealed that one out of five supermarket shoppers makes an impulse purchase at checkout. Although getting a pack of gum at a counter won't leave you broke, but incessantly purchasing items that are not on your budget will make you measurable sooner than you expect. The impact of impulse shopping on your bank balance is not immediate. However, at the end of the month, you will see how much money you have been throwing away on unnecessary items. Research shows that impulse spending is worse if you are the type of person who moves around with credit cards. So, unlike broke individuals, the smart ones understand the harmful effects of carrying their plastic cards around, so they prefer to go out with cash. 6. Playing the Lottery Statistically, the odds of winning a grand lottery prize is 1 in 292,201,338,000. However, many people buy these tickets every week with the hope to strike it rich someday. Amazingly, Corley stated in his book Rich Habits that 77% of people who buy lottery tickets are those struggling financially and only a small amount of 6% of wealthy people play it. He concluded that the reason for these records is that those who are wealthy do not count on luck for their wealth. They work hard and smart. But for the broke ones, they assumed that someday, luck would shine on them and they would have their big bang. Hence, even when they lose, they continue buying lottery tickets. If you have the opportunity of meeting a baker for the first time, do you ask him for bread or you seek to learn how to bake bread? A lot of people stay broke because they will choose the former option. A distinctive attitude between the poor and the rich is that while the poor are salary-minded, people with a rich mentality are experience-focused and network-oriented. Unless you go against the urge to always ask for something of immediate pleasure, you will remain broke. If you want to break free from misery, you must be open to learning from others. Network, build tangible connections and commit yourself to mentorship. Bear in mind that there are three things you can achieve from a job. Contact, experience and salary. Salary has the least value. 8. Staying around broke people Have you heard of the story of the little eagle in the chickens? I'll tell you. A newborn eagle found himself in the company of chickens and before long, this eagle began to act like the hens. He could not even fly. One day, it came in contact with another eagle and realized that this new creature was its kind. This new found eagle, which was mature, became the hero in this little eagle's soaring story. The moral of this story is simple.  The easiest way to stay broke is to hang around broke individuals. Likewise, if you want to be rich, be in the gathering of the rich. Their lifestyle will motivate you to be wealth-minded. It is hard to get ahead financially if you only associate with individuals who have bad money habits. 9. Borrow money to purchase depreciating assets Here is a scenario. Paul wants to buy a car to ease his transportation problems, but he does not have enough money. So, he takes a loan and purchases the car with the hopes of paying through a period of 8 months. A few months after getting the car, he loses his job. His only means of livelihood. Well, he still has to pay his debts, so he decides to sell off the car. Guess what? The car he bought for $3,500 now costs $1,500. Taking a loan is not a bad idea. In fact, most rich investors are debtors. However, when a loan is spent on liability or an asset that does not appreciate, it becomes a problem. Many people remain broke because they are stuck in paying debts for things that are not profiting. It is essential to understand that building wealth is not a game of luck. If you want to break free from the cycle of being broke, you must be aware of the behaviors that are holding you back in the first place and work towards breaking free from them.


And I'm Rob. I am Rob. Hello, guys. Today we are, this is a session on metrics that matter. This is a beginner session. And it is essentially work that both Raj and myself have done at a pretty highly regulated, in a highly regulated federal environment. So we're going to share with you some information today. So. Well, I would say that there's still one set of measures and metrics that have been put in every domain that, perhaps, through this discussion, we need to expose to the things that work for us, perhaps, that we need to focus on. And somebody has told me a wonderful quote that they just came from a section where they were all about, don't drag anything. Is this fine? Is this something that I want? And the context, I want to keep it in line. Very, very large federal programs, lots of regulations, Congress and more and so forth. So we have to have some different way of attracting our work. So anyway, so that's the context and the agenda. So first and foremost, as we get started, I'm going to ask you guys, what has done me in today's lean, agile, DevOps-inspired world? We are, of course, in a DevOps strategy. What does that mean? Shipable. Shipable. Shipable to? Yeah, all the way to production, putting code in the hands of the customers. And what are the implications of that notion of being done? What are some of the implications of saying, I am done when my code, from idea to customer, is in the hands of the customer. It's shipable. What are the implications of that? Yeah, that's all my theoretical stuff. Value is the only thing that matters. It's in the hands of the customer. Feedback, all that good stuff. But the implications from our delivery perspective, testing and operation perspective, the implications are we need to be sure that when the code hits the customer, how is it going to react? Is it really ready? Software development inherently is anonymous, becomes interactive. There's a lot of uncertainty. Now when your definition of done within our DevOps context is code in the hands of the customer, the questions really are, how do we minimize some of those uncertainties? There are uncertainties, of course. Now that I've said this, that's my quote. Is this today's one of the biggest delivery could be, is this one of the biggest operation? So one way to perhaps minimize all these uncertainties, maybe, is through some way of measurement. Measurement is essentially about collecting data, observing a certain problem faced to, as it draws an insight. Measurement cannot solve the problem for you. All it can do is check some light in a certain context of all the game. The insights and the actions determine how you're going to react to those measures. Just a little bit of a very close contrast here, because often these terms are misuse and misrepresentative. Measurement can come in two forms. Measures are concrete things, usually quantitative, and they apply to one thing. I have five apples. Number of production data, these are examples of measures. Metrics, on the other hand, are usually comparative in these. Redaration, you're essentially targeting two different dimensions, meaning, looking at the same measure, I have five apples. A metric, maybe, I have 10 apples more than yesterday. Or in terms of incidents, instead of just being a concrete number, a number of incidents, a metric, maybe, number of seven incidents since my last trip. You're always playing more than one dimension, and you're talking about a metric. You're talking about a metric. Now, when you talk about metrics, as you mentioned, you are, in lots of sessions, you're talking about metrics of pastures that are easy. So, with your perspective, how many of you are doing some sort of measures? These are measures of metrics in our organization. And, lots of both, right? So, are they all evil? Can you give me an example of evil measures? What are you talking about? What are you talking about? It's targeting an individual. It's targeting an individual. It's a necessary evil step. But, there's lots of debate about metric being evil. Can it be gained? Absolutely. Metrics can be misused. Metrics can be gained. Don't talk about it. No matter what set for a metric being evil. But, that doesn't mislead me. Metrics are all bad. It's in the context of how you do it. What's an example where you could potentially gain a metric? All we have is a new team. Core coverage. Core coverage! Thank you so much, right? Instead of this high bottom line, say that, you know, I'm gonna fail my bills, if it doesn't reach 95%, you need to start it. What do you think is gonna happen? Extra round of likes. Yeah, or just to focus, a certain true, a certain true, a certain true, right? It's very easy to gain numbers. So, there's a certain way you can use these metric, where it's not that it can't be gameable, but it thinks that you're gonna try to use them in a manner where it's actually better to the organization. So, this is some of the challenge that we face, and we'll sort of walk through some of the methods and how we try to make sure that it wasn't gained and not be misused. So, a good, so, good hard claw is a lot of out there. It really says that a measure becomes a target, it seems to be a good measure. What good hard claw is really saying is, if the measure itself, the metric itself becomes a goal under itself, you've completely lost its balance. And this is a powerful concept that he mentioned and something that you ought to keep in mind, right? Your test coverage is the end goal, it's useless as a metric, right? It's a metric towards something a lot. And that should be the context for all of our, as we design our metric for our users. So, having said that, what are examples of, so what makes for a good metric? In the first one, I did what makes for a good metric? You said you get an all metric, some sort of thing. What metric means that, I don't want the actual metric itself, what was it about the measure that made it more valuable than some of the metric? It makes sense to learn. What makes sense to learn? Makes sense to learn, okay, something that's helping us to give you some sort of, yeah. It generates insights. Generates insights, yes, that's the point of metrics. It generates insights. It generates insights. We generally identify a potential problem. Indicate a general area of moral disturbance as they call it, right? We have to find a key element in that. So, simple, they're together on a goal, yes. So, some characteristics of a good metric are, first and foremost, is that you can understand. If you require the PhD to decipher the metric, it's sort of useful, right? And especially for leadership, you want metrics that make sense, because oftentimes there's a tension between the development team and leadership who can support us, but also the metrics have to help cater both means. Comparative metrics. At least we find that metrics are a ratio of some sort. They're always trying to balance or measure against a different dimension. Maybe it's over time. Maybe it's compared in user support. Always ratcheting up tension between two dimensions to us feels like it's a much more relevant measure than a plain number. So, as an example, I think I have 10% more unit of coverage than last spring. That's perhaps to be no meaningful, isn't it? Right now, our discovered number is 32%. That doesn't give me the context as to in terms of how we're progressing or we're progressing, you don't have to, right? Comparative metrics are more useful. Last but not least, the most important criteria when designing metrics and doing metrics is, is it going to change the behavior of teams and individuals? This has to be one of your primary criteria when you're beginning to start choosing your metrics. So, having said that, I've looked at examples of characteristics of a good measure. I can tell you from at least the experience that we've had is, you can have all the critical metrics that we wanted to measure, but if it's not aligned towards some larger business goal or outcome, it will completely, meaning that you have to start identifying what specific business goals or outcomes that we're trying to reach and then, design your metrics because then they will provide you the gain. Are you progressing towards this given goal or are we sort of like that, right? This has to be your starting point when you start talking about designing, all right? So, having said that, there are some guidance that we'd like to share with you in terms of what are the three, the key dimensions across which you might want to design your metrics and how would these various dimensions sort of completely each other as you're designing the metrics and will go through some of those dimensions and see metrics. Thanks, Raj. So, as Raj stated, I'm sure most folks have seen this before moving from essentially left to right from a pipeline perspective. We've tried to organize this because this is sort of DevOps Day here at Agilindia. So, again, we've tried to provide, again, a topology to go over some of these key drivers and metrics. We'll break this topology down into some of these really four success factors and dive in again on some specific metrics that you may take home and reuse and hopefully stimulate some further thought. So, first would be, as Raj has already stated, would be our business success, this being one of our drivers, right? What is the success, meaning, you know, exactly, if we look at this image here below, and a lot of new companies or startups or whatever else, there's a lot of impetus, a lot of action just to do something and then something will happen afterwards, right? So, I'm going to build some awesome thing and something will happen. Typically, there's a lot of sort of afterthought and forward sort of back looking. Again, to Raj's point, we want to think about what are we trying to measure from the business perspective? What objectives are we actually looking to measure against? So, it could be something like a market share or it could be something like new business service enablement. These are sort of key indicators of success from the business perspective. Again, should be tied to some sort of measurable objective. Again, too often, a lot of companies, even mature companies, as we've seen in certain projects or initiatives, they fail to actually tie any sort of metric or measurement to what they're starting off with. And essentially, there's, again, in phase two, it's a big question mark, what happened? What are we actually going to accomplish? How would you get there? So, here is actually some data that we like to share. Again, the data we're going to show you here are examples that we've derived from, again, a pretty highly regulated federal environment to help illustrate some of these factors. So, this is a pretty simple, straightforward example. You're all familiar with, or not, right? I'm going to start at, this is just an example of what I'm talking about. And our goal, just to go back and bring this slide, is we had to, we wanted to increase your digitizing of a bunch of paper forms in the developer sector, and our business motivation was really to digitize as much as we can with a high quality, because for the general public. And the question really was, we had 50 developers in one specific project on a shared code base, how do we really track the work that's being done? We thought of it as a basic one-up chart. But what we noticed there was a different interaction by our leadership as well as the team in terms of interacting with this one-up chart. Yeah, we have a different number of stories. We have our sprints, and things are happening. Each sprint goes with things up. And we just, what we realized, this is not really helping the office in a whole lot. All it was doing was getting over it as more data and so on. We took an example from Spotify, the music people service, and that was essentially to, for every time it produced a one-up chart, we also, in our program area, answered three quick and this was live data all the time. The big questions were, when will all the planned stories be completed? And the second question is, what is the total number of stories that's going to be completed? What number of stories will be completed by a given date? And the third one is, can we deliver all the planned stories by a given date? If you look at these three questions, the first question is asking the question of fixed scope. When are all the plans for each going to be completed? What's the scope of the question? Based on the trend lines, you guys have all been there with your one-up chart. We just say, okay, it's going to be done in spring nine. The second question answers this. It is fixed date. If I have to deliver something by a given date, what will get done? So that's a fixed date, like the scope question. And again, based on the same trend line, we're able to answer, we can get about 120 stories from the story that we've done of a given date. And finally, the last question is a question of fixed scope and fixed date. Usually, management is interested in this sort of stuff. I would also, and I want to analyze this data. And clearly, based on our trend line, no, this is not possible. So making little adaptations to simple charts like most of us are familiar with, actually completely change the dynamic and our conversations with management. I think that this is simple for them to understand. So again, it's burn-up chart and burn-up chart. And this is also a really good conversation. Okay, we can't get done, but what can we get done? So meaningful conversations, just by making these other changes. So again, just to wrap up on business success. I mean, we're talking about business outcomes, identifying what those key indicators of success are. And they could be varied, depending upon, again, the maturity of your company, maturity of your product, and they can actually drive certain, again, from key indicators, they can actually drive how you're going to shape, which way you're going to move, as Roger's already stated in this example, to, you know, if it's a new company, you're probably going to be focusing more on stability and getting things rolling. If you're sort of a more mature, you may be focusing on more efficiency of your resources, being more innovative. Again, having some of these outcomes with your key indicators aligned properly can help drive the future state. So the next sort of, from the archipology, the next sort of driver here is customer slash user success. Again, this is, everybody has had, I'm sure, had some level of dealing with a customer, has had some interface or action in dealing with service desk responses. Some of these results here could even be, from a DevOps perspective, getting data from our customers, for example, from our AB test results, actually putting something out for specific testing, canary releases as well, receiving some sort of feedback from our customers. Customer ticket volume is also a very good metric that we probably use pervasively across, not just from the development perspective, but again, from the operations perspective. We have to actually tie, or actually have to rely on a new group or a team to receive these types of tickets or requests, and then ultimately, we're the, from a development perspective, we're the folks that end up having to deal with fixing those challenges. This is something we want to track, maintain, and ensure that we have the capability to continue to enhance and make those volumes either decrease or responses decrease. So the data that we're showing here again, this is from a pretty highly regulated federal environment. As you can see, we've got pending, this is improving responses to our issues. The challenge was we were having a lot of tickets come in on this certain project, and we were, a lot of the tickets were coming in, they weren't being resolved in any sort of manner that was quickly. We ended up stating to the actual development teams, as opposed to proxying this off to some tier one support crew. Devs, as you move forward, you guys are going to own your work, this work as it moves to production. Not only that, but you now are going to put a pager on your side and on your belt, and you're going to be tier one in some sort of manner. So the dynamics changed for these folks working and having to reschedule themselves in a queuing perspective, setting up schedules so they were able to actually respond from a front line to any issues. As you can see from both of these products, as we progress with you build it, you own it mentality, you can see our pending dropped dramatically over time, almost becoming completely equatable to having almost near time resolution of those tickets. You shed light on the talk today. Can we have more capability in isolation, not worrying about the manifest defects downstream? So this is of course a lot of pushback from the teams that are already trying to have them go, right? Sure. But also if it's not maybe enabling a couple of aspects, improving that, perhaps we've got to do something about it. So by the same token, we have to do some defensive part even though, you've got to build all the new features. Oh, and by the way, you're also exposed. You don't support for the features that you're putting into production. You'd have to, they have the capacity of this capacity. You'll have to make sure that you're balancing this kind of work along with the future of what they're doing. And that's the self-management. And again, if using some sort of data to illuminate that this is what the team is doing, if you want to improve customer satisfaction, perhaps you've got to write it down and decide if we're going to take it or not. Again, from efficiencies, from staffing, hiring the right talent, these are all indicators of success. So there's a lot of different metrics that you can move forward with. I'd like to take the opportunity of something I've read recently. This was a Google in their discussion of a customer reliability engineer. This is something that's sort of spun out of their Site Reliability Engineer book. And it's about customer anxiety and the ability to try and reduce the customer anxiety and make it zero. That's something that we all sort of strive to. And then just reliability in general, from a, as we move forward with, from our talk this morning, migrating into the fine world of multiple containers and multiple objects, as well as service providers that we now rely on, the customer anxiety reliability becomes very important for customer success. So just thinking about how we can actually get some level of metric from our customers that would be speed back to ourselves becomes very important, again, from a service provider like Google Cloud Compute or AWS. These things become critical. So I'm sure lots of folks have dealt with tire fires and trash bin fires in the world of operations. Who's, anybody done operations in the past? Awesome, so that's a small handful of folks, awesome. So 3 a.m. calls in the morning, always a fun time. This is our third sort of driver here. And as you've seen from our, I've actually put in deploy release and operate. I'd like to say that operations now is, these are areas that they are also responsible for. Again, as we move forward. In sort of a historical context, or maybe a classical context, a lot of sort of operations success metrics can be derived from network, platform, latency, things of that nature. As we move forward in relying on more cloud providers, CSPs, more objects to look at, some of the metrics we're actually trying to capture are starting to change. They've changed and they've also grown. So from a classic perspective, let's say we actually have like two nodes or two hosts. The typical amount of metrics would be maybe 150. Now let's say on those same two nodes, we have 100 containers. The amount of metrics essentially would be over 10,000. So you can see that again as we move forward, metrics and the number of metrics we're starting to receive is growing dramatically. And the same sort of hardware. So again, this is data that we've pulled. The impetus of this data was the driver was actually trying to actually have from our main line, we're actually trying to have something that was deployable. That was the goal. So what we actually tried to do was start to track just the simple success or failure as we were doing our pull requests and starting our actual build process. As you can see, there was a lot of challenges up front as we started to increase the number of jobs we had over time, our failure came up and we ended up having to sort of look at other data to figure out why are we having this, the number of jobs are increasing from our deployment success rate, but we're having a lot of failures. What we ended up doing was actually going in and diving into different various test suites. And we found that a lot of our regression tests were kind of bloated, they were failing, there were challenges. The manual testing, we essentially had to rip out. So again, diving into other data, other information forced us not only to sort of readjust how we were moving forward with our pipeline with this goal in mind, but also sort of refactoring our regression tests. This ratio, number of fail deployments. So this really helped us doing the problem by looking at a number of data points that were directly telling us this was a problem but it helped us sort of investigate close up and isolate the problem. So again, back to our metrics pipeline. Again, from an operate perspective, I'd like to illustrate, including deploy, release, and operate, included some metrics here that we've found very useful. Again, I think really the purpose of this slide is to illustrate, from a DevOps perspective, the ownership of looking at these metrics, investigating these metrics has grown from not just looking at, again, latency and availability and my pings is something up and running. We now need to take a step, a broader scope and look at what is happening in our releases. What's happening in our deploys? What level of predictability do we actually have a gain? How is that deployment frequency? And then matching that up with our development success. So this is sort of a potentially a port mantue from a metrics perspective of where development and operation will come together. So as we know, or as a lot of folks should know, when we talk about development success, develop, QA, deploy, or the three main themes from a metrics perspective and a topology perspective, it's always sort of a, when we think about DevOps, I think we've heard something that's shippable. I think it's something that, so typically that infers just like speed, vast, quick. We tend to leave out the quality perspective here. So development success, as we stated with develop, QA, and deploy should invoke and include just as important, good to high quality. Now, a lot of times as we've seen and probably illustrate in certain contracting environments, there is reward for speed and zero reward for quality. This happens all too often. So essentially having some of these metrics in place that are highlighted in a meaningful manner to provide feedback to managers of those contracts to help adjust what these folks correctly do. I don't think that's a normal feature. As we indicated in the previous slide, the VR deal in this range is not easy of a project. You have hundreds of developers committed to what they'll call the main item. So it is the challenging environment. And we provide this visual chart to everybody and in the common area for leadership and teams which, pretty much, it affects how the teams are utilizing their capacity. And as you can see, not all of them, just about cranking out new features and cranking out code. There's technical depth concerns that needs to be addressed. I don't need to touch on that either though, so we'll get back to you for that. Maybe artificial improvement, new code. And in our case, because we're moving towards the use of this, your own functionality, team line support, that's the key, the use of all the dealings with the projects that you get to support. And you can see, we are dealing with lots of productive problems. It can be, that's there, but how do these metrics in place at least provides the visibility to both teams, as well as leaders so they don't keep pushing for the things, you capability or whatever else your context is, by having how the capacity being allocated around the most powerful in terms of management and team understood that there's only, whatever capacity you have, you balance it against it and does that easier to deal with. And this sort of stuff, unfortunately, we're in an environment where we have a strong leadership pushing for dev ops, and all of it, how does it not need to support, again, it will be experimenting with the kind of things that we've been doing and talking about. So again, just to sum up, I mean, from a development perspective, we're, as we're, from the portmante of being responsible, ideally, to all the way to production, we wanna help emphasize that these key metrics and drivers to include development, QA, and deploy. Now, from a traditional CICD perspective, these metrics provide a lot of data to the dev team as has been illustrated and also provides them the ability to have more of an operation mindset as they move forward. So they can be responsible for that deploy and they get feedback immediately so they can do something about it. So I think we, it was Wednesday, we attended Evan's session on part of his no-project experience, and really his goal was really focusing on business outcomes. So Raj and I, just for S&G, said, hey, why don't we take a pass at this and see what this looks like? So this is, again, I'll give Evan all the credit here because this was done during his talk and we modified this a little bit. The goal here is to try to provide you guys like, again, I was talking about a metric, it needs to be ideally tied to some sort of business object. So our outcome title here is we're trying to provide a common platform for CICD, a platform as a service or a container as a service. Our measure here is team adoption rate. So we're kind of, from a owner perspective, we're taking this from the provider or the landlord of this service, right? We wanted to ensure we have some sort of, we're tracking adoption rate. So our current baseline is probably one team a month and we, because we are not having any targets, we're abating the target piece and so how do we actually measure team adoption rate? What does that look like? So Raj and I sort of needle around and figure that probably features are important to folks. Form factor is probably something we want to keep track of and the frequency. So below is just a little bit of sort of mock data that we put up and as you'll see, sort of what makes this two dimensional again and makes it a metric is the date. We want to keep track of the date. Event being something as simple as a log on or actually doing an image stream into this pass or cast system and then the form could be their IDE or maybe they're hitting the GUI to do this type of thing and then the actual count. So again, just a sort of representation of how perhaps you guys as you move forward, looking at what business objective you've got or outcome that you're trying to achieve, just simple exercise to move forward with the try and draw out some metrics that'll get you there. Oh, well Raj, so were you part of that session? Yeah, yeah, so we pulled the metric out or we pulled the target out because at this stage where we were trying to accomplish, we didn't want to have a target. I mean, a target could be very, very important. I agree, especially if it's tied to some sort of financial aspect. But for this, we're just want to measure our ongoing adoption. So tying it all together from all of the context that we've got really starting from business success to the other tale of customer success, the two key pieces again that tie it together really is the development and operation success system. Again, balancing our speed and risk quality cost, setting our key indicators of success, again, reducing time to obtain a response to our customer. So we've actually aligned, this is kind of a takeaway slide that hopefully folks will leverage to take advantage of a bunch of different metrics that help promote some fall. For metrics, again, there's a lot of restrictions along the way, but it's just about 20 cents. But make sure that they're all in challenging different drivers and insurers. Why do you want more people to adopt our testing protocol? So it's a lot of log-in decisions of making sure there's always a lot of time to do it. So as we talked about earlier as well, with this, as we move forward with new technology, as we move forward with the ubiquitous movement to more virtualization, more cloud containerization. Again, the number of objects we need to actually track and monitor and obtain metrics from is increasing, then it's very complex. And then if we couple it with all of our other success areas, that's a lot of data. And this is a framework that sort of the adaption or source of this image is from the Yard of Moderator at Turnbull, James Turnbull, it kind of hacked it a little bit with the customer success areas to provide where this data is coming from. Again, this is really, we're looking at data sources, we're looking at sort of a router or a transport mechanism and how do we are aggregating all of these events, again, that being quite large. And then what are the destinations? Is it just the storage? Maybe it's just security data or log data that needs to be stored away and maybe looked at at a later date. Is this something that we need real time or near real time action against? Is it alerting data? Is it thresholds that need to be set off? Whatever the case may be. So again, this is a framework that's been well used. A lot of folks such as Etsy and Spotify, even Google have their own manifestations of this framework and essentially what that framework ends up looking from the display perspective in an alerting perspective. We're gonna show you one that's from Capital One Labs, it's called Higia. And this is again a representation of multiple data sources providing metrics and essentially a single dashboard pane of view. So we're looking at data. Again, this is potentially sort of your CICD dashboard and the health of what's actually happening on your pipeline. So we're looking at with the top, we're aggregating data from JIRA that's giving us our sprint information. We're getting actually our Git or GitHub repo information. Again, Jenkins information. Any sort of static code analysis that's going on. Potentially performance data and deployment data. So again, we have a good snapshot that's easy to consume. Again, coming from multiple disparate sources. Yeah, this again is from Capital One Labs. The source here, you can actually download this. It's open source tool. Yeah, you can. Yeah, if you already, not only that, if you have these tool sets already in your pipeline, it's very simple to have them plugged up. Yeah. You can also create your own tool, your own sort of widgets and your own data sources and integrate them into this tool as well. Whatever makes sense. So, you know, a few minutes left, but if the few takeaways is, as I mentioned, there's lots of messages out there. But I've mentioned, it's useful, if it uses the right context, if it engenders some sort of behavior change, it's useful, as is code from low-rises. Measure me in a logical way, expect logical behavior in an open source. So, if this is talking about talking about velocity, velocity, velocity, give me more story. Oh, I think it's gonna happen. I'm gonna just make my head 20 points if it's wrong. I'm gonna get high velocity. But when high is that covered, that's all I care about. Here's what's gonna happen. I can write lots of write-offs, lots of bonus tests and then increase coverage. So, that's important. But don't be intimidated. There's lots and lots of semantics out there, think into what makes sense. And think of all sorts of semantics that's targeted. I mean, that's what we did, if you continue to add new metrics as well. And never target individual.  Make sure the measures are across the different dimensions. We've presented the four different private dimensions all the way from business because they're coming up with that. The metric is never about one thing, it's always about balancing multiple drivers. Make sure you learn metrics in that fashion. And then, I would strongly suggest that use comparative metrics. Why do we try to compare this? We're adding up attention across different drivers. You provide a deeper insight into the real-life function of the project, which is important. And finally, make it with nobody interacting with it, it's sort of useless. You want metrics, you want them to be simple, and you want to display and have leadership and deep interact with it. Without it, it is pointless. Finally, just a couple of things we're just starting out. Start with the business outcomes that you're after. Take one or two metrics, design those metrics, let the data provide the insights, and the insights themselves will generate the action. Metrics by themselves are like good friends, George and Ricky is around here, and the point of metrics does not indicate the problem. It's to just shed some light in the general area of disturbance that you have, and then use those insights to drive action. That's it, thanks guys. This is what we did, right? First and foremost, we had to make a technical data track, right? First and foremost, right? It was a developer, I, they were complaining about it, but nothing was really being done about it. First and foremost, we just listed all the technical metrics, items that the team had, and then once we had that, we started to see what's the level of effort for trying to address that. We then balanced the value proposition against the effort in terms of always going to take that particular issue, and once business determination, we made sure that the technical teams with the architects, their technical depth items were balanced if you do the team back off, and they would kind of be allocated to start with a different, that's the fact. Okay, thanks. Yeah, because they're, I mean, the product owner doesn't know what they're talking about, but it's our job as a technical team to start the company and doesn't do that. So that's what we're doing. Make it concurrent, estimate the work, and then make sure it's in that one cycle. The product effort was being the company that we do at midnight, it is work, and we're thinking about it. Yeah, yeah, sure. Again, this is something that came out of the Google Site Reliability Engineer work, and October of last year, they started talking about, again, a customer-reliability engineer. The idea is they wanted to actually have more success and a partnership with their customers on Google Cloud. So they brought up this concept of, again, a customer-reliability engineer, and some of the things they wanted to start talking about was anxiety. What is that customer anxiety? And their goal was to minimize, if there was a score, they want to minimize the score to essentially zero, and again, increase the partnership with that customer and see the success. To the point where I think they're experimenting with Google and the customer, if the customer is willing, having a partnership is the fact that when they're ready to do a release, they'll actually go through like a PRR together, and it would be then up to Google to say yes or no. Thank you. Do you have something to say? That's interesting. So the question really was, a lot of the data is coming from the development team, and the concern is they're asking you how you're going to do too much. So in terms of the measures that we took, it really is all in our chair, so it wasn't really asking teams to do additional work on them. And we have approaches of those teams for the long-term, and some after the long, collecting that data from their Sierra, and then that product doesn't really, they're really not asking teams to do any extra work. If anything, it's for the team to protect the team, but they can make their work as far as we're adding their work to them. Well, I just want to make you elaborate on some of the factors that gave you this data, and any data that can be managed by the team. Yeah. So let me give you an example of that metric that is, oh, by the way, are we ready for it, right? Yeah, last one. Okay, so let me just give you an example of the test coverage one, right? So if you start to set high watermark, this is what we did. That was a big mistake, right? We said from now on, we want a high test coverage, whatever the percentage is, 25, whatever. Yeah, that was a big mistake, right? That was a big mistake, right? We said from now on, we want a high test coverage, whatever the percentage was, 25, whatever. And in hindsight, that was a bad way for us to go about it. The failing will say, if you don't beat 85. So the adaptation we did instead was so setting a high watermark, we started where we are, meaning right now we're at 40% but what we said is we will never let our coverage cross the low end. And over time, we wanted to make sure it was constantly changing. So it's really the same data but changing the context of how we're looking at it, right? So setting high watermark, we never let it cross the low where we are and in time, we want to make sure that 40, maybe my sprint, that's inside the 45% so we constantly push the needle further and further up the stroke setting the goals that are really setting up the cater and driving the wrong kind of behavior. Does that help at all? Well, thank you so much for your time.


Presbytery compromised, massive hemorrhage, airway collapsed, and head injury. This was the second recon battalion's trauma run. This training was designed in the spirit of competition to enable all the regiments to send their best corpsmen and compete against one another in a standardized rubric. It was focused around combat casualties, specifically combat trauma. The goal was that every team would learn or refine their skills, and then from that they would take those lessons back and then in the spirit of competition have similar type of competitions in their regiment and their battalion.