All right, we are three minutes past the normal starting time. So let's just get started. Thank you, everyone, for joining this morning, afternoon, evening, depending on where you are. We are going to talk to you about a product that we worked on in the beginning of this year. That's called RPMotorspec. And when I say we, this is going to be me, Jérif Chibon, Sonata Spingu. And Niels, if you want to introduce yourself. Yeah, I'm Niels Philipsen. I'm called Niels. And we're in the same team as Pingu. And what we're talking about, we've worked on in the first quarter of this year, before all the circus with the corona virus started. I think let's get going. So we can directly move to the first slide then. Niels is going to have the ends on the keyboard, and what I'll be doing in the first part of it. So the first thing we wanted to go through was the fact that what we are presenting here, the prototype, as well as the IDs, Niels and I were sitting on the stand here to speak about this. But this is not actually just the two of us. Just not, I'll rephrase. This is not just our baby from the two of us. This is also something that Adam Saleh from our team worked on. And it's also the result of a lot of discussion we've had on the diverse list. A lot of people have given their inputs and thought about this, including the Incompire Climb, and many other people who haven't listed here. So the first thing was, well, thank you for everyone on contributing to this. What we're presenting here is a solution. It's probably not the solution. It's a step forward, hopefully not a step backward. There is already a second proposal that addresses some of the same angles that was pushed forward by Nicolas Mayot as an F34 change, if I remember correctly. So you'll see there a second way of doing something. You'll see here another way of doing what Nicolas is presenting, or you can see Nicolas' proposal and another way of doing what we are presenting here. So let's start with the problem. So the problem is we currently have in spec file, we have two fields, the release field and the change log fields, which are manually maintained. And this leads to a number of problems. One of them being that every time you have a polyquest that touches one of these fields, the polyquests are going to conflict with one another. That also includes outside of polyquests that's doing merches from a branch to another. If you don't do a fast forward merge, then you'll often have a conflict in these fields. That leads to a decision where every automation that you want to build on spec file and automating spec file, cleaning labor will be more fragile because of this field. They will need to be able to handle reconstructions or change log and release. And let's be honest, we currently have three change logs in Fedora. We have one that is described the change of the spec file itself and that is in the spec file. We have one that described the history of the git repositories that stored the spec file and that's the git commit log. And we have one in body, which is meant to be user consumable. That's something which shows in the GUI for updating a system. And that is the body date notes. Now, the truth is that for lots of packages, the RPM change log and the git commit log are just about the same for 90% of the commits. The one potential difference that you won't have in spec file is the usual, oops, I forgot to change the sources file commit that you will see in the git log, but not in the RPM change log. And but for the vast majority of packages out there, outside of this, I forgot to update the sources file, these two change log are going to be about the same. So the idea that you're trying to go to, and you'll set your clue for the next one. There we go. The idea is that we remove these two fields or we don't remove them, but we no longer touch them manually. So we want to automatically bump the release when we do a build. And with this, we actually want to keep in mind to the upgrade pass. So we want to be sure that if you do a rebuild, if you rebuild the package for FN minus one, you can still upgrade to SFN easily, as well as generate the RPM change log. And then since we believe that it's going to be most of them, there will be most of them similar between the git change log and the RPM change log, the idea is to generate the RPM change log from the git commit messages. But then we run into the questions of, yes, but not every commit message should be included. I forgot to update the sources file. It's probably one thing that you don't want to have in the RPM change log, so in the spec file. And we also want to be able to edit and correct. Oops, I made a typo. I referenced the wrong bugzilla bug ID kind of thing. So we want to be able to exclude some commits from the change log, as well as edit past change log entries. And we want to do this in a way that does not impact everyone. We want this to be opt in. So you don't, if you don't do anything, it works as it does today. If you are interested to test, then you can opt in and see how that works. So there is a question already in the chat about the upgrade pass. Is it really important anymore? It is a good question. One of the things is that the DNF system upgrade now does a distro sync rather than an update. So it will downgrade everything that is lower in the new version. So it is technically less important than it was, but it seemed to be, it seemed for us to be a good practice nonetheless. So there are two aspects. It is a good practice nonetheless to try to keep FN plus one at higher version release than FN. And the second thing, it is something which we currently still try to do. And I don't think the packaging guidelines have been amended to say, you don't have to care about the upgrade pass anymore. So we still want to try to adhere to these constraints. So with these problems and this idea in mind, we're starting to look on how we can do the RPM change log. Ta-da! So we looked for a few ideas. One of them that was actually given, I believe Jeremy Klein is the first one that mentioned it on the dev list. It was about using git tags. So the idea is when you believe a comment is ready to be built, you add a tag to it. And in the metadata of the tag, you cannot find a message. And that message could include then your change log, could include the things like also the release field. Some people, the secrecy could be stored in there. But that doesn't actually solve the change log problem, the three change log problem, because well, you will still have the change log in body, so we can ignore that one. You'll still have the commit logs. And now instead of storing the change log in the spec file, you store it in the git tag. So it can be auto-generated, but then you still have to go and check it and clean it up the way it is. It's not actually removing it, it seems potentially helping to create it. But at that point, Fed package build, Fed package commit as an option that just puts what's in the change log, whatever you enter as a git commit message. So technically that option is basically already there. Plus it's gonna be hard. How do we want to retrieve that information? Should we include a thermal-like syntax in the git tag to extract, okay, this is going to be the release field, this is going to be the change log. Then there is the old question, now do you edit such a change log? Do we allow magic keywords that says, well, this tag is actually replacing that tag over here because I made a typo and this is what you should be now using. Do we want to use git push minus force, the dash force for git tags? Do we want to allow people to remove and add git tags as they feel? That seems very potentially problematic because then you may end up having two packages with the same change log that actually come from two different commits and they don't really, you lose the history of what happened or why this build was including that change log because in the meantime, the tag that was included and that change log got moved to somewhere else. So that was a solution that we didn't really like. So we looked at another one. So that was a song I did that was shared on the devil list and that's something which seemed interesting. You generate the change log from the commit log up to the point where the commit touches an external file that is external to the spec file at which point you just include that file. So you have a change log file in this git and just plain the change log just as it is today in the spec file you just copy paste that into an external file in this git and from that commit and all the commits above the more recent ones, this will be auto-generated from the commit message and as soon as it will basically start from the most recent commit and move down but as soon as it faces a commit that touches that file the algorithm is gonna start which means you can edit that change log file and add all the new commits and reward and typo fix everything and then it will just include that file and stop trying to regenerate anything from there and that's the idea that we liked most so that's the one we went for. Then we started looking also to the release fields. So one of the idea and that's I think Neil brought this from the DOBS angle experience so the open build service that open SUSE runs that already has the feature of bumping the release that we built was to use the number of commits and number of builds in the release fields. That was very tempting because it's very easy to you change the version and then you count the number of commit that was done since that version was changed and then you can ask the build system how many builds have there been with that version and that number of commits and then you can just increment. That's very easy to do. Two things that we didn't know exactly how to handle and that were not happening in some ways. One of them was the merge commits. So when you have a single branch linear history that's very straightforward but when you start to consider merge commits that goes that branches and get merged. Well, that's becoming a lot different to handle. How do you count the commits? Which commits do you count? Which commit do you not count? How do you handle that? And then there was the old question of the upgrade pass because that reflects to the merge commits. So if you do a push to a commit on the right branch and so you get number of commits one and then you merge that, you met that branch into federal 32 branch but then there is a merge commit which means suddenly the one commit that you have added on the right becomes two commits on the federal 32 branch which means suddenly the upgrade pass is broken. So that is something that we still wanted to maintain as I explained earlier. And there are a few ways we could do to address this. We could start with the disk tag at the front. So then it becomes version, dist, number of commits, number of builds. It could be one of the one we were looking at most was to try to do version, number of commits, dist, number of builds to try to mimic more of the current behavior of things. So there was a few things and we did not know if we wanted to go in the past of changing the place of the disk tag and the entire distribution and now our tools like DNF package kit although I don't think this one is maintained so much anymore or just would be able to cover them. So this is still something which may be doable. This is something which we didn't, well, we started looking at other things as an alternative. The second alternative we looked at was the ArcPackage Utils approach. That is a utility that Climb wrote which essentially adds a templating engine on the top of the spec file macros. This would work and Climb actually has it working. It does the work, it does the job. It's just we did not like the idea of having a template on the top of macros that seems like a fairly fragile approach. It's adding a template on, it's adding layers of complexity and layers of potentially something that could end up being fragile or hard to debug. So that's also something we could work. It does work, Climb has it working, but it is not the most appealing idea. So we looked for another one again, there we go. Yeah, so the third idea we looked at was basically to try to use the build history and the build target to mimic to the human maintainer. So to reproduce what humans currently do, just using the build history and the build targets. I'm going to touch upon quickly the build history. So there are two ways we can do that. One way would be we just create code G and tell us how to behave. But that means that you need network access every time. It's gonna be hard if you have a slow internet connection and you need to create the entire build history of C or something like the kernel that will take some time. So that's the place where we decided to use the git tags as a mechanism to record which commits was built and then using that information of git tags with the build history together with the build targets which you can provide, which Isaac Koji provides or you can provide as a as a packager when you want to build locally. Well, then we have sufficient information to be able to make our logic works. And so that's the idea we went for. Nils, if you wanna... Yeah, so that's the idea we went for. We are trying to emulate the human, the current packagers experience as good as possible. We do realize that there will be cases and spec files which will not be able to be supported by Appliance spec, Autospec at this time. We do hope that this is a step forward in the right direction. We do realize that it's pregnant going to be the final answer that shall be craving to stone and never to be moved again. So the way we do that is you have very, very little changes in the way we work and how you opt in. Opt out is also fairly easy. We produce reproducible RPMs. So the spec file itself is you build a spec file locally and in Koji you may end up potentially with some differences. But once the SRPM is built in Koji, you can rebuild this SRPM in any system that is RPM based and you'll get the exact same output all the time. So the spec file is no longer the unit of reproducibility but the SRPM become easy. We tried to prevent the upgrade pass as much as possible. That was one of our goal and that's what we are trying to achieve with the emulating the humans. And we do support local builds from the spec file. It's just there are a few tricks to do but basically it is there. So to give you an idea now easy or hard it is to opt in into this. Well, this is a very simple spec file. You have the name, the version and then the release becomes that macro and the changelog becomes another macro. And that's all you have to do. But don't you if you add an existing changelog you will want to save this into a separate file which I believe is simply named changelog and the existing release and you can basically ignore. So how does it work a little bit more in practice? RPM motor spec itself is a CLI tool. It's a Python module library that comes with a CLI tool as well as two Koji plugins. One that runs on the builders and one that runs on the hub. So to give you a little, we have a small graphical representation of the workflow. When you do the Fed package build Koji will start by building the SRPM. The building the SRPM means we're basically doing a Git clone. Then we have, this is the Git checkout. Once the checkout is done, it's going to, Koji is going to call all the Koji plugins which are post SCM checkout that includes the RPM motor spec one. So RPM motor spec will do its magic there. And then we just Koji will just build the SRPM as it does normally, which is the equivalent of RPM build dash BS, I believe. RPM motor spec is going to pull from the Git repositories, the existing tags that we mentioned. Once the SRPM is built, Koji will build the RPM as it normally does. Once the RPM is built, Koji will tag the build as it normally does. And once the tag is done, Koji will call all the plugins that are post tag that includes the second RPM motor spec plugin. And that plugin will call this Git and say tag, add a Git tag to this specific commit which I have just built successfully. And that is the way that the Koji actually manages itself to record into the Git repositories the build history that RPM motor spec then we use in the build SCM spec. And now I'm going to leave the floor to Nils who will run you through a little bit more information on how that works. So I hope you all see the next slide. Okay, the quant line tool is basically the thing doing all the dirty work. It reads Git tags, which get there by some mechanism we'll talk about in a moment. The commit log messages and this to the site changelog file where we put the history in or where we can edit things after the fact. It comes up with the next release value for potential next build and generates a missing changelog entries that aren't in the changelog field yet. And it has code to fill in these values in the spec file like when later this one, the Koji when it creates the SRPM there's where we hook in. We ensure that the latest builds are tagged in this Git. So everything is prepared for the command line tool to run. And then we use that command line in the build route to insert the things in the spec file and then build the SRPM from there. And this one is going to create the SRPM and then build the SRPM from there. And this one is then used to build the final packages. Okay. Then Pingu talked about that before after a build was successful it just tags it in the disk repository. We are implemented a small API endpoint on the Pegasite for that so that Koji doesn't have to have its own Pega user to do that. Okay. So we have a couple of known side effects and shortcomings. So that's always a good topic to talk about. So the order commits came in or the order builds were done is how the change look is sorted, not the commit dates. I mean, anybody who's worked with Git for a time knows you can shuffle these around by rebasing or cherry picking or stuff like that. So the order of dates doesn't hold too much importance there because I could cherry pick a very old commit from some other branch and we don't want to resource that. And we can also at least with a prototype while it was still working in staging we kind of have a couple of problems with staging right now that's related to the color move but when staging is back in order again you'll be able to play around with that. So right now you can just tell Koji to build from some random commit and this will of course influence like let me backtrack that sentence doesn't make too much sense. We can adjust, we have the tip of a branch like let's say Federa 32 right now the current latest stable branch and we can do any number of builds from the same commit with that scheme but we could also build an older commit from that branch and that would then like the change like history would look a bit weird there. I mean, let me just, there's some more to that. I'll post that in the chat. The nitty gritty details about that are here about both of these points. Okay, so right now the prototype has some shortcomings in that it's a normal Python package that is it's in the version Python path it's a normal site package and if you upgrade the Python version on the builders suddenly it wouldn't find the package anymore and the command line tool wouldn't be functioning and stuff like that. So the solution to work around to that is that we do it like in the olden days and make it like have a copy of it and use a share RPM order spec something that's not tied to the Python version and then this should work regardless of the exact Python version. Like if you go at it with a new Python interpreter it won't care that it's a new Python interpreter as long as it's compatible but Python has been good to us in that regard so far. We haven't gotten around to pre-release to implementing pre-releases that we put quite some thought into it and we made some preparations in how the macros work like if you think back to the example spec file the auto rail macro understands a couple of flags and options for these cases but there's no code behind that yet so we would have to implement these use cases before we can use it for these use cases. And in order that the spec files are buildable locally we have to have the macro files available on the packages machine. Right now it's, you would have to install it manually. We could just install the dependency from the Retter RPM config package to our one or integrated with that one that doesn't make too much of a difference. And all the other tools are currently totally unaware of the magic we're doing here so we need to integrate with them a little better. Right now you would have to tell FAT package to skip NVR check because it doesn't really know the right NVR because it would have to have the, it would have to know to talk to our tool for that. Okay and scratch builds from local SRPMs don't work it like if you commit to your repository if you push it to disk it and then tell FAT package to build a scratch build from a known commit hash that works but from an arbitrary locally produced SRPM that doesn't work yet. And we would have to enhance FAT package to create the, to pre-process the spec file accordingly before producing the SRPM from which to produce a scratch build. And with that, I'll hand it back to Pingu. So our current next steps are basically we used to have the staging environment so before the column was started we had deployed this on staging and we were able to test it there. That's also how we got the feedback but for example, the tie to the regular Python packages which makes our PMO to be currently not working with the Python stack. So that was a very good feedback. So we want to deploy this again back to staging once the staging environment is back online so that we can call for testers on this again. After this, we do want to, you know, we have had a few people telling us that this is worth bringing to FASCO. So one of the point of this presentation was to give some more explanation on what the RPM to spec does, how it works but also asking you if you believe that this is worth pushing forward. If so, then we will bring this to FASCO for consideration as them if they believe if they agree that this is worth pushing forward. Then we need to fix the information shortcomings including things like the RPM Dave pump spec change that we need to do. That was discussed in the chat here including making Fed package aware of auto release, macro, including, you know, fixing the pre, what's the, not pre-commits but help me here Nails. Finish the macro, the pre-release and post-release use case. Because where the other use case we haven't addressed yet. So. Okay, so fixing these and then of course to use all the fixes and improvements. But that's basically what's on our roadmap. The first step being today, do you folks think this is good to push forward? And then the next question, the next slide is basically if you have any questions. And I see Yann as one in the scratch wheel section. So there is Fed package has two ways to do scratch builds. You can either do Fed package builds dash dash scratch which is going to do a, which is going to call Koji and ask Koji to build from a Git URL and make this build a scratch build. Or you can do Fed package scratch build dash dash srpm which is going to generate an srpm locally first, upload that one to Koji and have Koji do a scratch build from that. The issue is that the Koji pre-processed the spec files you know, at the srpm level. So if you do a Fed package scratch build dash dash srpm without pre-processing the spec file locally manually yourself first, then you won't have the spec file will not contain the macros that are needed for the RPM to build correctly. So the way you can do it currently is that you will have to call the RPM to build manually on this Git repository before you do the Fed package scratch build command. Or and the proper fix is going to have to teach Fed package to call RPM with a spec on the Git repo, on the Git repo before for spec file that uses these macros called RPM with a spec before when generating the srpm before putting it to Koji. Does that make more sense? Okay, thank you. Another question, what about two levels of macro use auto change log or change log to change log plus previous change log entries? Okay, so the question, Carl is basically wanting us to get rid of the change log macro entirely from the spec file and just have the auto change log instead of it. And have the NILS have an answer to that one. Contrary to looks change log isn't a macro. So yeah, I'm not sure if it's workable if it's workable like have auto change log to expand to change log and then RPM build understands that I wouldn't bet anything on it. So we'd have to try it out. I'm not sure, I guess it's mostly a matter of taste. If it works, if it doesn't, it's out of the question because then we lose the feature of a locally rebuildable package. NIL says that RPM doesn't expand any macros. Like if a macro expanded to change log it wouldn't work, do I get you right? Is packet aware of this change? Yes, the packet team is aware of the RPM auto spec work. We've had a few discussion with Tomasz Tomaszek. They were actually looking at RPM auto spec in the early days. Then they saw the discussion under the release and they were like, okay, there's still a working going but that's basically they are aware of that. And I actually think a little bit like NIL here that RPM auto spec would probably simplify their life to where we knew it was a question. I mean, if not right now you can always get all of us on IRC, wherever we frequent on the devil list, I guess. The project you're seeing on pagoda.io slash dora-infra slash RPM auto spec I've put the link in the chat if you're looking for it. And we do have one question for Jens about comparison to Nicolama use proposal. Do you want to take this one NIL, sorry? I think ours is at least for existing maintenance it's much simpler because it's really, really, really tiny change in your workflow. Your spec file before and after will look almost identical except the difference that the changelog is missing from the spec file. I think Nicolama's approach would you would have to adjust more yourself, your workflow more to that one. It's to be frank, I only got aware of this approach this week so I'll stop here because I hadn't had too much time to look into it myself. There was a question from what happens when Fedora messaging screws up and Koji fails to tag a commit with the build info and the answer is simply Koji doesn't rely on Fedora messaging directly called Spiger CPI. And if that was not available for some reason I don't know whether the build would fail or not I expect. I hope it shouldn't fail at that point. We should just we have to revisit that one. I mean, mind that we worked on that couple months ago so it's not all super present in our heads but the way we should do it is retry it a couple of times. If it doesn't work out, then fail. If you remember back, the builder plugin will ensure that the git tags are present. If that failed at that point then the subsequent build will fail of course but then there's something more serious. So what Neil says here is that basically one of the first thing the builder plugin does is it asks for Koji for the build history of the package and then it checks that history against what's present in the git tags. So it makes sure that git is up to date versus the Koji database basically and that's a second place where we actually are able to tag things in figure on this git from the Koji build history. We have a question from Ifa. Is there likely to be a follow on project that you know work from this work that you have already completed? And the answer is yes. One of the ideas that if Fesco thinks that this is valuable and we want to move forward for that then we will have to schedule a, I don't think this will be a three months initiative but it's maybe a one to two months thing where at least three people push this. And I also remember from earlier on during the talk a question about the actual macro names involved. I think right now this is a prototype so nothing with regard to naming is set in stone yet. So if there are good reasons to name it something else just maybe to also differentiate it from competing approaches we can do that still. It's not in production anywhere right now. So there's the question, could we consider making it so spec files that don't have a changelog change that do not have a changelog section entirely would automatically. We can do that. Change log. I would, the answer to that is everything is possible with time and money. I think it would even be fairly simple. And that probably would be a, we would instead of changing, checking the auto changelog is present. We would just check the change. Some slight change to the conditions involved whether to get active, but yeah. That's maybe a good idea. Do you want, can you open a ticket for this? Thank you. We are a few minutes ahead of schedule. So I'm more than happy to let you guys go to have the more social times guys and girls I should say to have a little bit more social times out there or we can hang around and give answer a question if there are any. All right. I guess we've existed the list of question here. So thank you everyone for attending. Thank you for your questions. I've only seen support in the chat. So I guess people are happy about this. So let's bring it to Fesco and talk to you if I had to schedule some time to finish this.


Hi guys, welcome back to my YouTube channel. This is Daniel Rossell here back to talk about one of my favorite topics which is backup and data protection and specifically using the M-disk an unusual option for those looking to Do their backup entirely physically not using the cloud And using optical media. So there are still people who are interested in this technology I think it's fascinating tech the M-disk as I mentioned before it's available in 250 and 100 gigabyte Discs it's available for DVD and for Blu-ray, although I think these days really it's only the DVD Sorry, the Blu-ray is that are gonna be of interest to people So I've been using the M-disk and an M-disk burner to back up three years worth of video content It was a massive massive pain, but I got there and I now have a binder full of M-disk But we all know of course if you're a backup fiend, you know that that even that Much effort is not enough because a proper backup approach Three two one necessitates that you're gonna have an off-site copy of your backup as well as an offs as well as an On-site copy because what happens if I have my wonderful M-disk video library and the house burns down or the house is flooded or Someone throws out for some reason my library thinking it's a DVDs from the 1990s So there you go. So that's why you need off-site in your backup approach now what I've done here using the wonderful draw.io software is I've gone ahead and Plotted out a few ways that you could do off-site and this is actually what I'm intending to do I'm visiting the US this summer and I'm going to be actually Creating a little off-site archive Bringing it with me physically in my luggage and leaving it there And I might do this process once a year for the here on after so here's a few ways I've thought about that you can if you wanted to do something like this you could So one way of doing a make this process a little bit easier if you wanted to burn So we're gonna need to burn two copies of our data. So I'm burning 20 gigs of video data at a time You might be burning photos whatever the case may be. It's not really important What type of data right now? I have not been able to find on the Marcus a burner with two tiers In other words, it's gonna have you're gonna be able to burn two m-discs Simultaneously, so I asked people who are also into the optical archival game and they said well what you can just do is buy two Burners two individual burners and hook them up to your computer at the same time and burn away So burners are not that expensive. They're about a hundred dollars these days for a m-disc capable blu-ray burner So if you're using a bunch of Linux It's very easy apparently with windows you might have more issues running the same Software it's simultaneously, but as I'm using Linux. So that would basically I just hook up to External blu-ray drives to my computer load into m-discs open the program twice and Burn at the same time and they should take about the same amount of time Now we all know burning optical media However great it is it's a bit of a pain in the ass So this is just one way I can think of to try make that process a bit easier Physically buying two burners Okay, so that's workflow one burn simultaneously was two m-disc burners workflow two is I have a typo here burn and then duplicate So this is what I would love to be doing now if I could find a 220 volt Blu-ray duplicator because I have 16 discs and I really don't want to load them one by one Copy and paste the data which is going to take about 30 minutes per disc Then burn each disc again, which is going to take about 30 to 40 minutes per disc going to be a royal Pita, so here is another way that you could do it So you've burned your on-site copies and you're filling up your archive whatever and what you could do to save yourself a bit of pain is Offline duplication and then move those discs over to your off-site location So what I'm talking about here. You might see this funny looking device these guys exist Unfortunately, they're becoming a rarer breed for instance this Plex copier one-to-one Blu-ray M-disc Duplicator copier tower these guys like you know when you see these things for hard drives nowadays. They're still very much used They can work fully offline. This one is done is made by the company called Plex copier You know given that this this market pretty small You're gonna see the same brands popping up again and again Double-click image to zoom in so that's what it is You can see it's a it's a you can you've got one drive here one drive here stick in your source and disc If you ever used a offline copier They're pretty and pretty intuitive and you just say please copy my M disc and you can go away and do something more fun This guy is $360 if Plex copier wish to send me a free M disc burner I will say thank you very much Plex copier, but until until that never happens.  And then you burn your off-site copy And the only reason to do it in this painful way is that you do get a chance to verify the burner was successful Check it out a bit then burn the off-site And you can do this one at a time because what I actually do is I delete the data on my NAS after I do this Which in retrospect maybe wasn't a brilliant idea But if you don't want to do that you can keep the data there and then you know do this two-part process Do this, you know a few months apart So anyway, those are a few methods that you can use M disc for a three to one compliant backup approach How would you get your onto your off-site copies to your off-site location? You could do what I'm planning on doing stick them in a suitcase. You could post them one by one You could use the mail and post them to like a backup buddy in your backup buddy post you his M discs You could drive them to a you could drive them to your office You know wherever you have an off-site that's geographically separate from your onsite However, you can get from a to b or wherever you want to get from a to b should theoretically work and just keep an M disc Library there and between once you've got your two copies your onsite and your off-site You know that you're pretty golden from a data protection Standpoint you can do 10 or 20 copies, but diminishing returns really sets in from a risk standpoint after two Once you've got onsite and off-site you're pretty much set You don't need to create four different off-sites and scatter them on different continents Although you could if you really wanted to hope this video was useful Thank you guys for watching M disc fanatics of YouTube and more videos coming soon from myself


Welcome to All Hands Update, here's what's happening around your Navy. The 75th commemoration of the Japanese attack on Pearl Harbor and other U.S. military facilities on Oahu is an opportunity to honor the courage, service and sacrifice of the military personnel present during the attacks. The theme of this year's commemoration, honoring the past, inspiring the future, is highlighted through events this week that will continue through December 11th. Take a moment of silence today to remember our shipmates lost on a day that will live in infamy. Dwight D. Eisenhower carrier strike group entered sixth fleet continuing support of Operation Inherent Resolve. Elements of the 22nd Marine Expeditionary Unit aboard USS Wasp continued precision airstrikes against ISIL targets in Libya as part of Operation Odyssey Lightning. You can read more about what's happening around your Navy at navy.mil.


We're back in San Francisco. We're at Moscone West. You're watching theCUBE's coverage of RSA 2023. This is day two of theCUBE's coverage. Really excited to have Kumar Ramachandran here. He's the senior vice president of SASE at Palo Alto Networks, and he's joined by Rex Thexton, who's back on theCUBE. He's a cyber leader at Accenture. Gents, welcome. Good to see you. Thanks for having us. So Kumar, you just came off of a keynote. You do a keynote on AI, of course, and network security. It's not a topic here, it's AI. So how to go? Oh, we're all seeing the impact of AI and ML on cyber security. I was on stage with Neil Boland. Neil is the CISO of MLB, Major League Baseball, and we were really having a robust discussion on how one AI ML has become absolutely essential. The reality is this, right? Many vendors, including us, we've been using AI ML to secure and to automate operations for several years now. Chad GPT has put it in the popular side guest, right? And what we will continue to see is a transformation with co-pilots, autopilots, whatever the industry builds. But the discussion we had really was if you take it back to the basics, AI ML and data science, only work is we have the right data, right? And the data really has to fulfill the three principles, right? You have to have complete data. You have to have consistent data, and you have to have correct data, the three C's. Complete, consistent, and correct. That's right. And if you have fragmented systems, which is how we used to build network and IT infrastructure systems, it's very hard to get that. So it's very hard to implement AI ML at scale. Whereas in SASE, you now have a unique opportunity where you can get a complete platform, your mobile users, your home users, your branch users, your SaaS applications, internet, cloud, private data center applications, network security, all of it coming together. So now you have the opportunity of delivering transformational outcomes to your customers. Explain that a little bit more, SaaS. So it's kind of the big buzzword right now, but it has meaning. So give it some meaning, secure access, service edge, what should we know about it? Yeah, so if you think about it, right? Pre-pandemic, when most customers were employees, but really coming to a branch office, accessing corporate resources. You used to put security primarily in the data center because that's where you're delivering your applications from. The pandemic really accelerated the transformation where people with digitization, much more aggressive adoption of cloud and SaaS, and then hybrid work became a real thing. We now expect to work anywhere. In this model, putting a firewall or a security stack in your data center alone doesn't cut it. You need a model where security is pervasive and can meet the user wherever she is. So what we did with SASE was we said we have an opportunity to reimagine how security is delivered. So we built this highly scalable multi-cloud service that's distributed globally. Any user anywhere, when they're accessing their application, they get processed with a full stack of security. The other interesting thing about SASE is the policy model. The policy model has migrated to a zero trust model. Previously people used to have the notion of I'm in the network, if I'm in the campus in the office, I'm going to trust the device to a certain extent, trust the user to a certain extent, give them network access. Whereas now we are moving to a model where you really want a policy based on, hey, I'm going to publish an application, here's my user, let me specify a policy. My user has access to X applications based on a device profile. User application device rather than worrying about networks. I think those are the big transformations. So Rex, I know when Accenture gets involved, it's actually a real business. Actually, with the one exception of my first quantum demo was that an Accenture thing in Boston, it was unbelievable, I learned a lot, but that was very cool. But so you guys announced a bunch of stuff, the SASE diagnostic and advisory services, SASE implementation services, SASE is a managed service, so you guys are seeing the opportunity here. Explain what's going on from your perspective. So one of the things we've seen is, as things have tightened up recently, that a lot of our clients are slowing down the pace and what we're trying to do with the diagnostic is create a data-driven approach to identifying the business case outcomes, so how they can deploy SASE and save money. This is a real business case that there's a lot of cost savings that can be deployed. A lot of people are using legacy architectures, running backhaul traffic through the data center, and so there is a meaningful business case that we've made in that. So that's diagnostic, it's the goal in where we invest some time leveraging the Palo Alto stack to go in and it's a little bit of some feedback. Testing one, too. Yeah, here we go. Someone said feedback is a gift, not this audio feedback. Yeah, it's not, I was a rockstar there. I love that line. Anyway, so the goal is to help clients identify that, and we feel if it's a data-driven approach, they'll believe it, right? If it's just a conversation, consulting speak 101, they're just like the Bible. If you can show them the real data, the user experience improvements, the cost savings, then it becomes more real. And then the implementation, obviously we implement the technology in the case, but the thing that's more exciting to me is this SASE as a service or SASE as a managed service, because what we're trying to do is help our clients scale this technology, right? And a lot of they can implement some core use cases, but they oftentimes have a hard time scaling it out to the enterprise. And so what we're doing as a people at this SASE as a service is being able to go in and help our clients do this at pace, have an ongoing deployment and capability so that they can scale out through the enterprise in a cost-effective fashion and doing it in a shared service model where we can provide scale. Because I think that's the one thing, obviously you've known Accenture for a long time, we can help clients scale and we can provide scale to this. And I think that's what we want to do because we are kind of stressing off the importance for our clients needs to modernize. Everybody needs to modernize them. We need to retire tech debt, consolidate tools. I mean, look what it takes to do now in a network, right? How many tools? That's right. A ton, right? And you can come in and leverage it into one platform, reduce the number of vendors that you have to work with and reduce the toll blow. It drives a huge business benefit. So you mentioned the business case, Rick. I mean, it's hard dollars these days. It's very hard dollars, it's not soft. I mean, they want to see it. And so, and how has that changed?  Are they looking for fast payback? Are they looking for low risk? You know, low risk, high return?  Kind of what Cloud did was everybody ran real fast but their operations cost exponentially. And they want to be able to scale with keeping their costs flat and do more with less operating leverage. The labor operating leverage, yeah.   So we see four really important catalysts and projects that our customers are doing, right? During the pandemic, we saw this VPN replacement. Where people were traditionally having these VPNs that brought your traffic back to your data center and then sent it out. And if you're sitting in your home and you're trying to access SaaS services or internet in addition to your corporate resources, that doesn't make sense. So VPN replacement to a ZTNA model where you move to the app user device-based policies without VPNs, that's a huge driver. The second big driver really is your on-prem SWIG or secure web gateway replacement, right? People build these proxies that they used to deploy in their data centers, process the traffic there and then send it to the internet. Again, makes no sense. People are migrating to a cloud proxy. That's a big driver. Third big driver, SaaS adoption. People were deploying standalone CASB products to protect their SaaS applications. What they quickly found is that over 50% of the SaaS breaches are related to poor posture management. Because think about it, right? If I'm an enterprise, I can't understand every one of the 100 SaaS applications I have and how exactly I should configure it. And then every time the SaaS vendor doesn't update, what are the edits I need to make? So having an automated drift prevention mechanism, mechanism to protect it properly, that became the third big project. And then the fourth project that is particularly exciting, we're seeing the return of the branch. During the pandemic, rightfully in many industries, the branch office was less of a priority perhaps. Now it's back because people are back in a hybrid work model. I'm in the office three days a week working from home two days a week. So people are having to re-architect the branch. There's a very interesting thing we're seeing in the branch office. There's not really a very variable user population because people are in and out depending on day of week. People are using rich media and video and Zoom in the branch tremendously. People come to the office, some of their coworkers are at home. So I'm turning on 1080p video on my Zoom call. So we're seeing ridiculous amounts of bandwidth explosion and variability. These are the four big drivers for SASE right now. Are you guys, so you're back in the office too? A few days a week here? Yes, yeah. So it's interesting, I go in every day and it's like bookends. Monday and Friday, traffic's light except Friday afternoon, everybody's going out, right? And in the middle of the week. And so how are organizations, are they able to dial up, dial down? Do they have to? Yes. So if you think about it, if you're only deploying hardware boxes, then you would have to keep scaling those boxes for peak capacity, whereas SASE is a service you're delivering from the cloud so you can dial it up or down depending on how your branch is, right? The elasticity that you get in addition to security is absolutely massive, right? Yeah, I can't stress the security aspect enough, right? Bringing security to where the user's at, right? You know, it used to be impossible now it's delivered every day. That's exactly right. And you know, to echo some of the comments he was making about how Accenture adds value, right? Any IT project, there's a three piece, right? People process product. And vendors like us, we love delivering innovative and absolutely fantastic products, but partners like Accenture, those are the people and the process pieces, right? How do you ensure that you have the right kind of ROI? How do you ensure that the client's process transformation is adapting to this new model of security and network consumption? And they're able to accelerate that transformation for our customers. Well that's where you get the business value.  It might be, you guys have specific industry knowledge and like you said before Rex, you've got scale. What, how does this compare to some of the other, you know, relationships that you guys, describe the relationship over time. What gives you confidence that this is going to fly? I mean, you guys have a lot of experience. What are the similarities? What are the unknowns, you know? Yeah, so you know, I think the big, what do we bring to the table uniquely, right? What we bring to the table very uniquely in the marketplace really is that for all the use cases that I described, it's a unified platform. Now what does that mean? That in our case, it actually means that there is a single data lake that's collecting all the data related to network, security and operations for that mobile user, the branch user, the home user, accessing internet SaaS cloud data center applications. This has never ever before been done in the industry. There is absolutely no other vendor who's done that because we've actually collapsed that data. In one data lake, we're able to apply data science on top of it. And when you apply data science on top of it, we use traditional data science like correlations, regressions, we use AI and ML, we use LLMs increasingly. We're able to automate security. We're able to automate operational outcomes. So when a partner like Accenture comes, what they're trying to do really is get our customers to not only use one use case, but the more you use, you have one plus one greater than three effect. The customer saves more as they expand, right? And the best of the capabilities are native to the platforms. That's how the partnership, the customer benefits, Accenture is able to bring tremendous value. We're able to deliver an incredible platform out there. Well, Rex, that's your scale point that you were making before. What about from your perspective? We were literally talking about LLM and generative AI and the application of it to this because they have this, but we feel that we choose our partners that are innovative, that are not thinking about what's today, but what's tomorrow, right? I think that's what's super exciting to us. And to think about, there's a lot of work in creating policies, but as you learn from this data, you can auto-suggest policies, even auto-implement policies, and it can become super smart, because I truly believe there's not enough security professionals in the world to secure the world, right? And so we need to leverage these new technologies, but they have to be trusted. And I think with the data, the collection that they have, they will have a trusted, secure AI, which is super important, because if we get a lot of hallucinations, that's a big buzz word now. Yeah, yeah, I've been hearing that all week. I'm sure. But then people aren't going to trust it, and they're not going to use it. And so it's super important. That's why we need to work with innovative companies that are reputable, that understand these capabilities and these techniques to implement them in the appropriate fashion. And it's super important. That's a soil, come on, are you saying that's you? No, we implement, no, but we do implement. They're the ones that will run the LLM. They'll do the training, right? But we'll help interpret it for our clients and implement it in a meaningful fashion, yeah, exactly. Is that, I mean, again, it's a true partnership. It comes back to the business case. You're making a business case. You make a business case, you're making a promise to the client. 100%. And then they're expecting that they're going to get at least, and you've got to be, you have to come in with some degree of conservatism, and you've got to hit that number or beat it. It's like an earnings cause. You've got to hit the number. 100%. You know, where you going to get, you're going to get dinged. No, and I think, you know, I've made a living in this industry of doing what I say, right? And I think that's super important that you live up to your word and being able to work with partners and companies that you can do that is super important to us. And that's why we're working with. Do say ratio, got to be high. People oftentimes get that backwards. Say do, no, we want to do say ratios high. No, but it's got to be achievable. It's got to be attainable. And I think, you know, like I said, this is a similar moment in time, I think for security, we can really move the needle forward if we get this right. I think it is, I mean, you know, I think we're all old enough to remember some of these giant waves.  You know, it's very hard to predict what's happening.  You got all this GPT and foundation model stuff going on. And you know, a lot of layoffs and a lot of engineers now starting companies that are going to disrupt. It's kind of exciting. A lot of M&A going on and signing times. That's right. And you know, it's times of opportunity, right? And opportunity, if you think about the opportunity for the last 10 years, we kept saying, hey, data is a new oil and so on and so forth. You're seeing the opportunity right now. And you know, to his point, if you look at the security landscape, the threat vectors, as looking at some of the data points, right? You have, what is it? In less than 15 minutes, from the time of vulnerability is introduced or spoken about, you have scans coming into major customers. You have a 150 plus percent growth of the use of things like cobalt strike. It's not going to be too far away when you see LLMs being used by the bad guys. So if you're not using data science, you're not using AI, ML, to actually contract all these and secure your customers, your customers are going to be exposed. And that's why having partners who can guide customers into platforms where data is sking are going to be back. And I'm looking at this Unit 42 threat report. And 60% of the incidents take between more than four days to respond to. Yeah. Right. And it's 40% to zero to four. That could be three. And would you say it takes 15 minutes? So from the time of, let's say, some major company announces a vulnerability in their products. In less than 15 minutes, you start seeing major attempts, scans to exploit that vulnerability globally. Yeah, so four days, they didn't cut it. They didn't cut it, exactly. So you have to, if you're not using AI, ML, and data science, where it's LLMs, where it's deep learning, where it's ML, you have to use all of the above. But it all goes back to the data. Garbage in, garbage out. That was true pre-GPT. It's true post-GPT. So if you don't have platforms where data meets complete, correct, consistent, you're going to be out of luck. So what do you guys, go ahead. I was going to say, a lot of clients find out they have a breach from somebody else, which is quite fascinating, right? They don't even discover it themselves, which is quite fascinating, if you think about it. Well, hey, this has been happening all, for a long time in this industry. You go back to Stuxnet, all right. What about this event? I don't know, you guys here in 2020? Yeah. Yes. Okay, so you were here for the last RSA before COVID. I got the sniffles. No, just kidding. Yeah, yeah. Right, but it was a weird vibe, right? But now it's like back.  It's got to be at least 50,000. Yeah, there's a lot of energy, a lot of juice here right now. It's been pretty excitingly. The future of this conference seems very good. The future of the industry is good. I think the future of the industry is bright. I think we're at this, you rightly stated that we're at a very pivotal moment, right? If you take the thesis that data science, AI, ML, LLMs are going to be a critical part of this transformation, you really have to go back to the drawing board and say, the products I'm choosing in my environment, will they allow me to establish a framework that is not manual, that actually can be run with AI and ML. And I think that has to the start point of any IT transformation project, right? Whether in SASE or without, if you're not thinking about what is happening to your data, right? As a CISO or as CIO, you're going to be in a lot of trouble. For us as a vendor, our obligation really is to make sure that we are ensuring that we are gathering the right kind of data, we're very confident in that data, and then we're applying the might of our algorithms at scale, right? I'll leave you with a couple of factoids, right? Our system today, we see and stop 236 billion, that's billion with a B, threats every single day. That's over a trillion in a week, right? So it is that scale at which systems need to operate, and it's also that scale at which you start learning enough that you can actually implement things in line and security, right? So while the bad guys react in 15 minutes, in our case, 95% of the time, zero-day threats, threats the system hasn't seen before, we're able to stop using inline ML. So our reaction time is zero seconds, 95% of the cases. That's real time, yeah. That's right. All right, gents, congratulations on the launch, and good luck with everything. Thanks for coming on theCUBE. Thanks for having us. Thanks for having us. Appreciate it, you're welcome. All right, keep it right there. Dave Vellante will be back with John Furrier. This is theCUBE's coverage of RSA 2023. Go to siliconangle.com. We got all the news. We got multiple journalists and writers here. So check that out. We'll be right back right after this short break from San Francisco.


Good morning everybody. I want to talk about tests today. Essentially I want to talk about three different parts, starting with a little bit of background. What is a unit test, what is an integration test and all the theory behind. Then how we write unit tests and integration tests in KDE and then how to integrate it with a built system and with a CI. I hope I'm fast enough for a lot of questions. Okay, let's start with the motivation. Why do you want to write a test and I mean automatic test, which is something the CI is executing and tells your result? If you ever wrote a manual test and you had to test it and run it again and again after each change, you'll see it's a time improvement. If you have something automatic that you simply automatically start and see that you did not break anything if you changed it. And it's even better if you have a lot of tests that test tiny assertions to ensure that well to find errors easily and fast. It's also important if you ever refactor your code, you can see with your tests that you don't get regressions or you can you know where your code regressed and you can directly adapt to it. And some less commonly motivations, but both quite important to me is often mostly if you write libraries, you can use tests to document behavior, which is sometimes written down, sometimes not. And yeah, it helps you to not break unwritten contracts with your users. And another thing that in my opinion is extremely important in writing libraries. When you're writing a test to test your library, the most integration test you're testing your API by yourself and you see, well, it's working, it's usable or not. There are different kinds of tests. I will look at them to view angles. One is the angle of white box and black box. It's like the light is on, the light is off. When you are writing some piece of code and you're really in the code, you know, there's something complicated going on and it can break, then you're usually writing a white box test and writing a test that fixes exactly this situation to ensure that's correct. In a black box test, the view is different. You look from the outside. You look from the features you have on your library or on the API side or the features of the user and you see that the core functionality is working as expected from the outside. And it's always good to remember that there are really two different view angles for that, because you're testing different things. Another way to view at them is to look at what you are testing. So-called system under test. The unit test usually tests a class or maybe two at most three classes that are really tightly coupled together. So these are the red blocks here. It can be the application code. It can be in library code and you test it singularly, really small thing that you want where you want to ensure the functionality. On integration test, it's in my view, mostly these black lines. So you're testing the API, the access of a black box thing, like a library from the outside where you're only using the API of the library. So usually in such a test, you should never include any CPP file. You only should link to a library and check that's working. And then we have a subsystem test where you mostly look from the user perspective, that's what the user want to do is working as expected. A nice thing that should be named in such a talk is test-driven development. If you never heard about it, you should Google it. In a nutshell, it's a way of developing code where you start with writing a test before writing any code. And then you check that the test is really failing. And then you implement the code to fix the test, and, well, just the test. And if the test is just barely passing, then you write the next test that is failing again, and you're writing again the implementation code to the implementation that fixes the next test. And so over time, you're creating more and more tests, and you ensure that the code quality improves, because you're testing it really deeply. Usually, you also have a better architecture because you have to design your code that is testable. And often, it's a better decoupling of different code units, the introduction of reasonable interfaces, where you split code parts and structure it to really have a good architecture. And mostly, with test-driven development, you have to refactoring during the whole time of writing your code. So refactoring is part of your development, and so it works really nicely. Coming to how to write a test. In KDE, I think nearly all, if not all tests, we are writing using Qtests. It's a really lightweight framework, which is provided by Qt itself. The idea is you have a Q object or Q object-divived class, and every slot of that object is interpreted as a test case. You have to add some magic behind, so you have to add a main function that's automatically generated usually, but you're writing the class, you're defining some slots, and they are executed. Qtests also provides you with several test markers, for example, something to verify that the expression is true to compare if two values are the same and also comparison methods for a lot of internal Q types. You have a signal spy to check that a signal is really triggered and received, and there are a lot of more things. For those, you should look at this Qtest list of all markers as functionality, because there are some hidden gems that really make your life easier, and there's a good introduction how to do it. But let's look at some example how Qtest looks like if you never wrote one before. We simply have to include Qtest, then create a class called a simple class derived from Qobject, and I defined two slots, one I called in a test case and one my test. In a test case, this is actually a special slot because it has a special slot name. Here I added a list of those special slot names. There are a few more, but these are the important ones, I think. In a test case, it is always ensured to be called before any other test case, and similarly you have cleanup test case, which is a case triggered after the last one, and that helps you to set up your code and to clean up. Also for the individual test cases, you can define in it and clean up, which set up any test case. In this test case, I added two really trivial functions. I verified that two is true and that one is one, and so this test, if you compile it, should be probably true. Going a little bit more difficult or more advanced features you can do. For example, if you want to check inside any of these test slots that the signal is received, I created a small code snippet where you have a push button. The push button has this clicked event that you want to see if it's really admitted and received. Then you can create a signal spy, which is really similar to an ordinary connection. You should always check that the signal spy is valid because it checks that the emitting object is not null, and if you have string-based connections that you did not make any typo, then you emit the signal and you just check if there is a signal received, so it's count of the signal is one or not. After that, you can even take the signal and the parameters and check if they are what you expect. One important point here is here it's good to remember the training from David if you have been there Friday because the signal here we know it's a direct function call and so it's received before the spy.count. You should be careful because sometimes if you are testing some signals between different slots, you need different methods. For example, there's a spy.rate function that rates for some time and spins its own event loop to check if the events are received or not. Really, it depends if it's a direct signal call or a cute signal call. In 95% of the cases, it's totally valid to do this, and then you have sweating and you have to be careful. What you can also do with QTest is create data-driven tests. If you never do it, did it? Well, the idea is you have a test case but you are starting it with a lot of different input data. For this test, I have a slot that is called the same but with underlying data and you are defining columns and rows. It's really a table what you could think about it and you are fetching the tables or the lines of the tables several times and run the test with it. Another nice thing I added here is a QExpectFail to show it once that marks the next test function to be failing, which is sometimes what you want because it's not yet done and does not work yet. It's also nice because if you fix it in the future, the test will fail because it does not fail anymore and you can remove that marker. Another thing I would want to show is you can also test QtQuick. That's not that come, unfortunately, that you really test a QtQuick binding or QtQuick class. A nice way to do it and there are different ways. The way I prefer is that you start your own QQML component and you have your own QML engine. I have some test QML class and I don't show it here. It's simply a really small Qt object derived object there. It has property that I call test property and then I create an engine, I create a component and I say it's about to be created synchronously because I'm in a test and it's much better. I created, I see, did it create correctly so is there any syntax probably in QML and I verify that I have an object, I verify that it's loaded, which should be already done because it's synchronously and then I can directly access the root object of the scene that I created and test the functionality. With that way, you can look at properties and do even more complex QML operations inside the test QML class and provided we have properties back. Another way to do it is to use QtQuick tests where you have a QtQuick-based testing where you define QML functions, JavaScript functions with certain operations where you can define that you want to click at something and you test if it's really worked, but it's tricky sometimes, especially if you change dimensions and not take care on creating tests, but sometimes it's the real way to do it if you have a QtQuick. Small lessons that I learned over the years when creating a test, you should take care that a unit test, so real test slot is not depending on another one because it's much easier to restructure your tests and it's never a good idea to have them dependent on each other. You should also not test production code. There's usually a thing which you call a fake, you have also a stop and a mock, so it's test objects that you have behind that you inject somehow into your scene with expected behavior and expected answer that allow you to test something and that makes your test reproducible. You should also ensure that your test is not slow because nobody else will execute it then. You also should not test third-party code because if the third-party code, the library, is not doing correctly, you should not use it or you should fix it and provide tests to that library because it's still open source. You should not create a test that includes dozens of CPP files into the test case because then it's probably some architectural thing that you have to fix first because it will not be a good test in the future due to any refactoring you have to do. Also a good device when talking about tests, you should also look into software patterns. There are a lot of good books and they help you to write better code and better testable code and that then allows you to add stops and mocks and fakes mostly by introducing interfaces and doing other nice tricks. Okay, that was a really short run over what Qtest is doing and well how to get the test now into the build system that you can really execute it on your system. We are using CMake as a build system and CMake bearing CTest which is mostly a tool to execute tests. So you can use CTest to execute all of your Qtests and get meaningful results of it. The documentation is here. I will only show a little bit about how to use it, how to integrate everything into your application. It's the easiest way is to include the extra CMake modules at test module and then just use this CMake macro, ECM at test, edit there, define the sources of your tests. For example, the simple test class CPP I had before. The libraries you want to link to, for example Qtest, give it a name and maybe a prefix which we will see at the next slide while it's important and tell if it's a GUI test or not and there is some documentation how to use ECM at tests.  You can also tell that you want to have output when a test fails because you only get a list of results. And for the daily work the most meaningful in my opinion is CTest minus r and something and that runs just a test that contains that word and that's really, really, really important to just run the test for the code that you're running at the moment. And for convenience we also have MakeTest and that is just running CTest with all the tests and that's it. Okay and well not only you are running tests, also RCI is running tests so at build.ke.org you can see for your project except for the Playground project, test ones and they are one on different architectures. So they are one on SUSE and on 3BSD and again several suit versions and that gives you good coverage of systems to see if everything works nicely and if it's possibly possibly only one on your system because you have some test data relying around. A good thing to do today is if you have a project that you maintain just go to build.ke.org and check if all your tests are green there. That's really important to really have a look at the system because we have it and as far as I know in the future you can, you will have the possibility to get results even earlier directly in GitLab but that's an ongoing thing. Another nice thing you can use at build.ke.org is to see the GCaV coverage. GCaV is a small tool that simply locks which line of code, which possible code path is executed by your test and if you go there you see for all your modules which has a good test coverage and which not and I think the most important point here is to see where are the completely read corners, the corners where you could forgot to write any test maybe it's where we all test or I might got there but that's the point where you can see at improving it. Okay and that's what I prepared for now and I think we have some minutes for questions. Are there questions? Yes indeed we have. Thank you very much for this session. Applause in the chat please and we have a few questions so starting with in Ubuntu we spent lots of energy running tests when we build packages but whenever something went wrong the developers would say those tests are more for me. Is it useful for distro packages to run tests? In my opinion yes but we have to ensure in KDE that tests are always green and right now it's complicated because we have the split between pull requests and CI and I think it will be much better in the future when we have test reports directly in GitLab before we complete a merge. So we see everything is green and it should be green and at that point I think it's reasonable to tell packages that tests are guaranteed to be green and they should run it again and then it's still green. At the moment it's hard to for some projects they are always green for others not that's not good. Okay thanks. Next question. Do you have experience with other popular C++ test frameworks QTest, Cache2, DockTest and how do they compare to QTest? Not much. I looked a little bit in GTest but I'm quite happy with the combination of CTest and QTest except one point and that's the Qt Crater integration. In KDevelop we have a really nice integration where I can really run all the C tests differently in Qt Crater. It's really hard because the CTest test properties that you can also define to adapt your test behavior a little bit are completely ignored because they are only the executables that you can execute and they have better support for other test frameworks. Okay, thanks. Now someone from Python and Java background is asking for a good solution for mocking. Sorry, I don't have experience with mocking frameworks. Usually I write my mocks by hand. I know there's gmog that's used by several people. Sorry for that. Yeah, I think the person is asking for C++ mocking solutions. It's the same for me. I've not much experience with mocking frameworks. I write my mocks by hand. Okay, and I think we have time for one more. Does it matter if a test is reproducible? It's so important that a test is reproducible. If it's not reproducible then it doesn't make sense because sometimes it behaves that way, sometimes the other way. So you should really assure that a test is reproducible mostly by ensuring that there are not dependencies to your test system, that you set everything up by hand. That's not a good idea. You should set it up automatically. You should clean the build directory where you run the test and so on.


I've got to make my record. Come on, Diz. Look at your base. We're motivating the team, don't we? Yeah, I can see that one. Because there may be six groups of them. One win also, you know. One win. It's going to be a free activity. If you do it the first time around, you have to go... Oh! You did it the first time. Yeah, you did it the first time. Yeah, you did it the first time. Yeah, you did it the first time. It wasn't far off. It's too high. Do it, do it. When you do it the first time, don't touch it anymore. Yeah, exactly. Yeah, I don't do it anymore. Do you know him? I don't know. Yeah, I hear that. Oh! Come on, Diz. I've done it. For you recently, you've got some... Well, not even... Oh, that was close. What? What?  That's that thing.


Statistics and Excel. Bell curve. Test score example, part number three. Get ready, taking a deep breath, holding it in for ten seconds, looking forward to a smooth, soothing Excel. Here we are in Excel. If you don't have access to this workbook, that's okay because we basically built this from a blank worksheet. However, we started in a prior presentation, so you might want to go back there if you want to start with a blank worksheet. But if you do have access to this workbook, there's three tabs down below. Example, practice of blank. Example, in essence, answer key, practice tab, having pre-format of cells, so you can get to the heart of the practice problem. Blank tab is where we started with a blank worksheet and are continuing at this point in time. Quick recap of what we have done thus far. We've been creating our bell curve or normal distribution and we're not stopping building the bell curve yet. Why?