So hi everybody, I'm a bit surprised that the whole the room is pretty packed. So thank you for coming and thank you for having me here Today I want to talk about the mutation testing and the way we should we should actually take to to leave the stone age so a few things about myself I am a US developer by day and In my spare time, I'm actually hacking on compilers and this is like basically my hobby and This project actually my hobby itself so if you're curious you can find me on Twitter and I have two blogs where I write about some low-level stuff and LVM itself and mutation testing also and especially if you are into software testing and quality of software then I recommend you to Look at the last Link the system under test org There I and friends of mine. We are writing about different software how it's tested and so on so forth So it was shameless plug Yeah, this is the the outline for today's talk I will start Was the quality of software then I will talk a bit about the unit testing then we will move towards Mutation testing then I want to present the tool that I'm working on and at the very end I will try to present a showcase like a real-life usage of this tool, but I'm afraid that I will not be able to to to give you like some stellar Examples, but that's gonna be interesting by hope Yeah, so I do software for about like seven Yeah, seven years now, I think and I enjoyed a lot But one thing that I'm concerned a bit is the fact that software is just like broken like essentially everything is broken Like browsers compilers separating system games just anything it's full of bugs and it doesn't work and so on so forth So we as developers as computer scientists and engineers We are trying to put as much as possible efforts in four decades to improve the software somehow and there are some Some ways to do it. So for instance just just today. We had a great talk about formal verification of programs But I don't know much about this topic, but as far as I know We are not there yet. The tooling is is great It's improving but it's quite hard to to verify C or C++ because of this and find behavior and so on So another great approach is fast testing it Yeah, I think It's like here like for a while and now since we have better computational power we can apply it more efficiently So but I think there is no doubt that unit testing is the most widespread Technique that is used by software developers and yeah, it comes with the nice metric as well it which is code coverage and Yeah, today I Want to focus on just on this I'm not gonna talk about like flaws and advantages of fast testing or for Modification, but I definitely want to talk about unit testing and its problems So yeah, how it works in generally in general We have some code and yeah, this is like artificial example just to to to start So we have some functions some that takes two integers and returns a sum of them and for this function What developer does he writes a code like The assertion is that the sum of five and ten is more greater than zero, which is absolutely correct It's valid statement, but it obviously it's not enough because Five multiplied by ten also were greater than zero so if we run this test Then we see that zero tests has failed one test passed and the cut coverage is 100% percent So it definitely has some It might be misleading the code coverage metric might be misleading and I see that Sometimes people are striving for number for this number, but the code quality is Not the code quality, but the test quality is not as good And there is no way to measure it. Actually, there is no just no way how to assert how good our tests So here comes the mutation testing so I will try to explain the algorithm behind it like very briefly if If there is something like completely unclear or like really confusing just feel free to like to to interrupt me and dust I will elaborate So yeah, what we do like normally was mutation testing we have a program and we have a test suit We then execute this test suit against that program And every if everything is good if tests are passing then we do the following For the program we take a mutant we created mutant out of the program So we basically do some mutation on it and it must be some Semantic change the program should be different then we Take that mutated program and run test suit against it again and Then we check the result if the result is a failed then we claim that This mutant was killed This is essentially a good thing So if you introduce if you run a test against mutated program, then the test should fail If that's not the case then we claim that mutant is survived and this is kind of bad So it means that something is wrong either code or the test So here again our initial example was the test and the program so out of this program we can generate Many mutants, but I will just show like a couple of them So first one is we replace a plus b with a multiplied by b and the second one We replace a plus b with a minus b Now if you run the test against the first mutant then it is still Passes so it it yeah mutant is survived and it it gives us a hint that something is wrong However, the second mutant is well was killed and we are good to go here so Based on this example, we can derive your numbers so the total amount of mutants is two one mutant was killed one was survived and mutation testing also comes with the metric called mutation score and is calculated by the formula on the screen like killed Mutants amount of killed mutants divided by amount or like total amount of mutants and multiplied by 100% So in this specific case the mutation score is 50% and it's not good So again, you should not strive for 100% because you probably will not reach it anyway, but You can use it as just as a hint. So higher mutation score is better Yeah, few facts few things about mutation testing. Oh spoilers Yeah, sir so it was First proposed by Richard Lipton in 1971. I think it was called mutation analysis So these are the same kind of the same terms. So it was first implemented by Timothy, but in 1980s And I think it was for Fortran, but I'm not sure. So don't quote me on that There are several studies like quite recent studies that show that a certain mutated mutation testing systems for certain programs. They could help to find a box like Real false like quite high number of them like 70 and 90 actually 94 92% like to to be more precise but even Despite this fact mutation analysis was not or it was not widely spread because it has several problems as well as just everything so the first problem it generates like lots of data for just 10 tests for a small program you can end up with 100 mutants and if you Execute this like those 10 tests against those like hundreds of programs Then you end up like to running the program like 1,000 times in the worst case Second one, yeah, it's time-consuming because of this like the the data we have lots of data We need to process it to execute it takes quite a lot of time Languages they are not not all the languages I would say are friendly for mutation testing because It's I think it's kind of easier for languages like Ruby or JavaScript because you have just like a wall and well YOLO so but languages like C or C++ they are more they're not as friendly because You essentially need to to mutate the program like to change the source code like maybe AST, I don't know then compile it then link it together with everything and run and then assert it just quite slow Though I don't have real numbers Yeah, there is also a problem of a human test oracle so let's say we generated lots of mutants and We cannot just say like whether are they are they good or not? So we need somebody to actually look at those results and make the decision Yeah, it also comes with another problem, but I will tell a bit about it later It's actually and yeah another one problem. I was surprised by that I heard this like almost exactly this phrase like excuse me, but I write good tests. I Don't believe it and it for me. It sounds like excuse me, but my code is back free, which is well not the case My code is back free, but yeah, not in general Yeah, yeah, so the tool is called mal not mule so It with this tool we are trying to to solve some of those problems But not all of them and there are some things that we do to actually Improve the performance and make it well better in general. So we have kind of smart mutant selection it means that I Mean it just helps us to to decrease the amount of mutants and amount of data We need to process and it's very similar to you know the second point So we provide a means to control the data generation. So I Will elaborate on it. Yes, it's complicated as well bit So we also utilize the runtime compilation and jet particularly we use a low VM for that and it means that we We don't need to link something. We don't need to recompile things that much and we completely eliminate I owe bottleneck of I owe in this case so Because of that we are operating on the level of LLVM IR which gives some nice tool things about also comes with problems like You cannot sometimes you cannot Like correspond the IR was the initial code so they well they might be different especially with C++ with all these templates in lining and so on so forth and The great thing that it is language agnostic Given that all the languages are built on top of LLVM. So if something is built on top of LLVM, then it's likely that we can use it Yeah, so time to elaborate a bit on How the system works so this is the typical program we have like several source files bitcode files modules What not they have some functions and instructions and if we just go and Analyze the whole program as is then we will find out like many many many mutants many many places where we can introduce the mutation But it's not very efficient because because muscle like maybe not most but big part of this code is like just unreachable for whatever reason So the bet the better approach would be as what we actually do first we try to find the actual tests thing that are well tests essentially and Based on this those tests we take the call sites like call functions We are trying to build so-called cold tree and Based on this cold tree. It's like on this slide. You can see some modules. They are not needed So we can just eliminate them completely. We don't need to compile them We don't need to analyze them just I don't consume less memory and so on so forth so Yeah Whereas with this optimization it is actually still experimental It's a bit more tricky than I thought before and it has some problems, but it is it is definitely the way to go so for example in this slide We took some subset of tests from LLVM itself the target called IR tests and we just took like two and a half hundreds of tests and Before to actually to run this the whole system to process it We needed to look at like 400 modules and it took roughly Yeah, 85 minutes after this improvement like cutting those like unused modules We got like three times less modules and almost to it works like almost two times faster But again, it is experimental. It's not in master trunk yet, but we're working on it. So here is the most complex program like closer to reality and here we can see that on the left The the functions on the left they are tests and they go some way to like wires as some distance And it has some So for the same subset of tests we took distance to it produces the 100 1000 and a half Mutants and the Real execution time that I measured on my machine took like about one hour, which is kind of acceptable but for on the other hand if we took the whole program then The distance is 29 for for this group of tests and number of mutants It's I think there is actually back. It should be like orders of magnitude more like maybe two hundred thousands of mutants and Approximated approximation of like execution time is roughly 11 days, but it's very pessimistic. So in reality, it will take maybe seven days six So Yeah, that still doesn't help Yeah, and with with this numbers actually with this mean to to to change the distance to control the distance Even if it takes one hour, you cannot probably use it on your machine like daily basis because well It's kind of waste of time But it still can use it as like with the night night the bills for example And for Yeah, you turn control. Okay Yeah, there is another mean we can select some specific tests so let's say you have like 200 tests, but you are interested in like one or like some group of tests and Yeah, so basically just like select one test and work with it was like even like you reduce the amount of tests even further and I Didn't put numbers on slide, but the the I think the best case was like 30 40 seconds maybe for one test and the worst case for a group of tests, which is My 15 or 20 took like 15 minutes 20 minutes, which is like still reasonable and it enables like really really fast iterations So I I think this is like the the best improvement that we did so far the best kind of invention Yeah, so Few words about the system design how it works in general. So this is like in a nutshell The program consumes as an input a config file, which is just a yaml which is doesn't matter What it contains is the list of bitcode files Basically shows which modules where to get modules to to process and it also like has This settings for distance and some cash and so on so forth So but what the program speeds out is the another config file and the sql file so that another config file is Reduced version of the first config file. So let's say they initially you provide a config with 400 Modules and you found out that you were like used only hundred of them the next time you can run the program with that reduced config and You will not need to process like the 400 but rather 100 so it's kind of also improves the iterations Yeah, we also don't provide any so yeah, I mentioned the mutation score But the program at this very moment we do not provide any like short report like Your mutation score is something percent because again as I said, I believe that we should not strive for numbers We should just use them as a hint and what we provide instead of this is the sql file Which contains information about just everything and it has the advantages So if I run something and I got results I can treat them in any way I want and I don't have to restart the program again wait like hour or a week and Just again iterate faster So this is yeah the program from the like outside user perspective So internally it contains Consist on from several several modules one is kind of core. It has just a driver that Well controls the basic system. It has a reporter and it will be Likely extended or replaced in the future. So instead of sqlite you can report to the as like output or whatnot One essential part actually is mutation operators. I didn't talk about them before so I will just Briefly explain what's that so every mutation tool mutation testing tool They have some mutation operators. These are things that like kind of rules that Describe how you change your program. So it could be like replace plus with minus like remove void function or replace like a negate condition for example or I don't know like skip the whole loop for example or like anything you Can imagine there are also some studies that do the mutation operators for for Java. I think and they Go in further and they change the class hierarchies like they inject some classes remove and so on just to screw the system up even more Yeah, the second part is also like straightforward. It's a tool chain Maybe it's not the best name, but it just like jit compiler from a little VM and the object cache So when once we run the program once we usually don't recompile Biggest part of it with the next run. So it's also improves the speed Yeah, the most important in my opinion The most important piece the most important thing is the test framework. Basically, this is the last one thing now that is abstract because it can be Like Google test for example if you want to run the tool against C++ then we have a driver now and like all this infrastructure for Google test It could be also like exit as for example for objective C and Swift potentially and just I think two or three days ago We merged the initial support for for rust it. It's not production ready. I mean the whole tool is not production ready, but the rust You can just take it and apply on any project, but We'll get there Yeah So kind of showcase Initially like month ago. I wanted when I was planning the slides and so on I wanted to run the tool Against the LLVM itself and guess are some results that I went to it and find something probably something interested interesting but the problem that I faced that to actually assert the To actually make sense out of those mutations. I need to know the domain of the tool Very well, which is not the case. Unfortunately, I don't know that like the most part of the LLVM like AP Float and some other stuff But the results are available online They might be cumbersome and unclear and maybe cryptic because I am in this context and I don't understand everything there, but It's hard to understand like it's hard to know for me what people may need So if you are into it and if you are interested, you may take a look and give some feedback. I would really appreciate it Yes, so again the some numbers for example that IR tests Yeah, you can see here the mutation score is 43 percent, which is quite low and I actually saw Yeah, quite few places that can be improved But again, I cannot just like do it easily and fast. I need to to dive into this matter Which I didn't and the same goes for ADT tests. The mutation score is a bit higher on 66 percent But it is still quite low. I would say so what I did instead I took the one part that is Quite small and the code there is quite straightforward. So even I couldn't can understand it This the triple test, I don't know if you're aware what the triple is but it doesn't really matter. So just just some group of tests and Yeah, the mutation score is quite high, but I still wanted to see what's going on there and First thing I found is this kind of tests So we have triple we change the architecture and then we ask for Well, then we assert that some Like in this case like other architecture like based on that initial architecture is like exactly as what we need So what happened what's happening behind the hood under the methods like get little Indian art variant there is just like huge switch statement and that just maps things together and What we did we basically started like removing switch cases like one by one and we found that many Things they just can be removed and the tests are still passing. So in this case, it just means that We just need to add like to add more tests Yeah slides, I think switched but okay Yeah, so one could argue actually that this case can be found by code coverage and this is like absolutely true It will be there. I'm pretty sure but the nice thing about the mutation testing in this case that you get report like this one and you clearly see like What was removed like let's say you remove this like this case And you see like what what which tests were affected by this So basically what what tests you need to extend to improve to cover this case and it took me like just the whole flight from Berlin to Brussels like one hour maybe to improve this a bit and Yeah, so I just like added I covered all the cases switch cases that like in on cases that were not covered and yeah I committed this I think last night So it's in trunk now The second one is more interesting in my opinion So yeah, the test is like straightforward We set some property of an object and then we assert that property is exactly Like equals to what we just like used but this test actually will pass if we just comment out this line Just just because the elf is default value. So there is no I mean. Yeah, so what mutation did in this case Set object format the body of method was kind of removed like completely And the test were still passing. So also what I learned from this very specific example So if you have situations like this then it may make sense as a developer to cover like two cases At least two cases and then it's very likely that you're good to go and you are not asserting developed default value So again, I just like added like two lines and also committed it I Don't check for default initialize to not default in every test Can you repeat sorry now that you had a test case which test the people yes, and now you have a You have claimed this is true. Yeah, and then every all the other test case you initialized something which is not Yeah, this is how it should work, but this is not how it works. Yeah I'm well in this case actually you should have replaced the Mac O as first and the alpha second Because you still don't need the first set object format. So I think it's a bit trickier Okay, I cheated a bit Okay So the elf is not like default like is like completely default it based on so it's default for Linux Yeah, it defaults on Linux it but is different on OS 6 and even I run it on OS 6 machine the triple initialize to default value like it when you give the empty Empty string it thinks that it's Linux machine no idea why but that's how it works Didn't I didn't have this goal at all Yeah, so another example I it's not like the code exactly I think I found it in IR test, but I might be wrong But the idea is that you have something like if something then we use slow version of an algorithm Otherwise we use fast version of all the algorithm and we have some tests and if we basically introduce some mutation and we flip those branches, then the test still passes and again, one could argue that this is like Fine, but I would say no because the only notion that Something is slow and fast is like basically the name of a function, which is like not always the case there might be just some comment and It might be that it might be true as soon as like the developer like wrote this code in the first place But like next year somebody fixes a back in the slow version and it becomes not that slow Maybe and somebody else fixes back in the fast version and it became slow. So It in this case there should be some tests like to measure like performance how slow how fast maybe they are like not Maybe they are the same by performance and there is no need to write this code at all Yeah, I think that's pretty much this pretty much it about the showcase. I wanted to show So as I said the tool is still in progress and we have some open questions The first one is the most important. I think the most kind of cumbersome is the integration so since the tool is working on the level of a low VM bit code of IR we need to get it somehow some like from where from somewhere and To get the bit code for for tests from a low VM. I needed to use C make then ninja then Like bloody shell scripts was set grab and so on So we're just not straightforward if you want to use it on your project like plug-and-play. It's not we're not there yet So this question is still open and we are trying to To find the best way to do this to improve the thing. So the another kind of problem is you X It's still not clear again like how to do this like properly. So was What we did initially this escalate file and I wrote like bunch of ruby code to generate this like pretty and nice HTML and it really nice and it works for me, but I don't know how it works for other people and Again the question is still open. We are trying to find a way to To make the system usable for for people So another one I Think it's not not the question anymore since we merged support for rust We were talking about like which language we should try next because right now the system works fine with C++ But I am afraid that the system the current system design can be biased by C++ itself So we are wanted to take another language and to make the system kind of flexible and well good And there are obviously many many many unknowns So we don't use this tool in production yet. We don't have any users so far so if you think that you're interested into it and you want to try then I Urge you to contact me and I would be more than happy to help Like free and so on like just for for sake of the tool Yeah, so the project is available on github. It's open source. It's like we're developing it well in open way So if you have some questions or you want to get it give it a try and I will gladly provide your support for this And just drop me a line at alex at low level bits dot org Yeah, and for some updates You can either follow the project on github or follow me on Twitter and I am posting them Updates on the process Questions please Yeah, so I'm I think yes, so I found one one test it's quite lengthy and it has some quite like another lengthy function and There are many like ten assertions in this test And they all check whether the method returns true. So it doesn't check the like the It just checks like one one basically one case was different inputs, but one case. So Yeah, does it answer the question? Okay. Thanks Yeah, please So I Think it well it is compilable because we control this stuff So we try to make it compilable and We will run all the tests and all the mutants they are running in the they run in Child process and we control that process. So if program doesn't work well, it crashes Then we know that it's fresh if it if it happens to have like some Infinite loop then we also like we have time out and we just like catch this as well Yeah There are some debates on this in my opinion. It is good I mean the program is broken then so we broke the the program. So I think it's a good thing, but The opinions are where I Yeah, I don't know how Let's go from it, right So we we barely control the this aspect. We just Again, like make change and like YOLO. That's that's it. That's our approach Yes Say So we we are trying Okay, so the question is whether we can Just Yeah, so the question is whether we Introduce some mutations on IR level that are not possible in the source code level. Is it correct? Yeah, more or less so We are trying to not do this obviously we don't have any guarantees on this But we are well, we are trying to avoid some mutations We have some like filters and some heuristics to not do this So for example in case of C++ there are like lots of code is coming from the standard library for example it just was in line and it happened to be like the client code basically but if we mutated then it doesn't make any sense because It's unlikely that you have a problem in well In standard library that that's that's not what I wanted to say, but you get it. Yeah so There was another question I think yep, please So I got the impression that maybe for some projects if you want to get the better mutations card Maybe you start adding in order of magnitude more tests Do you have any idea on? How much pushback you would get then every time you want to make a change then for every line of code to change You haven't changed ten times more lines of tests and it slows down your project because of that I know the mutation score is good since good tests So again, it's good, but then those tests that he added should never be changed because they should be right And then if they're not it's his fault. Yeah, so I'm just wondering Any experience by anyone in practice, so I don't have any experience in practice that can contradict or Approve your statement, but it can be yeah, it can be so yeah the execution or that selection of mutants is this schedule kind of static up front where you generate Or do you do this as the testing progresses so you could for instance look at are we reaching a threshold in In which case there's no point in much longer Well, actually, you know, that's a good idea, but no we do so the phases like the first phase is analysis We guess are all the information about the whole system all the mutation points mutants and so on and then we start generating them and Executing so that might be actually Yeah, that might be a good idea Lobs Yeah Yeah, but there was like always the Trade-off. Yeah, you can miss something and yeah You should take a look and make decision then Exactly yes, yes, yeah, that's the problem why I didn't manage to give nice Analysis based on LLVM itself because I don't know many parts of it So I could not understand what's wrong test or code We have Sorry, yeah So Not not yet. Well, we do have plans like obviously, but I am they are like long-term not something that will do soon So thanks for questions


We've still kept our culture and our spiritual process. Don't say rural masses don't have spirituality, they are the ones who are really on. It is a common notion among people that spirituality is a rich person's area. So even if you look at the people who have gathered around here, we are all pretty much well off. And once we have earned for ourselves, we then find time to do yoga or meditation or look into our inner being. But as we were discussed all throughout the whole discussion till now, there are, yes, you yourself have said that 500 million people are still malnourished and there are so many people in this country who are daily wage workers, who at that point of time, spirituality and learning of their inner being would not be the most important thing for them. Their most important thing would be feeding themselves and their children for the next meal. Really? You think so? You do one thing, you go to any so-called spiritual event in the country, will you see rural masses there or people like you in Denim's? Who will you see? Yeah, yeah, yeah. So don't make conclusions which are not even true. Are we trying to prove something or are we trying to look at truth? The truth of this nation is, we have kept our populations in extreme abject poverty. Nourishment is missing, leave anything else. Nourishment is missing. In spite of that, I must tell you this, people ask me these questions wherever I go in the world. Sadhguru, so many spiritual teachers and masters, so many great beings have come in India, but why are we so poor? Why is India so poor? Where is the spirituality? Well, you know this, 250 years ago we were the richest nation on the planet. Everybody wanted to come to India. Why do you think of Vasco de Gama or that Columbus who made that mistake? All these people set off in the oceans that they did not know and over hundred expeditions in the ocean drowned, nobody ever wrote about them, all right? Because nobody writes about people who drowned in the oceans. So why was everybody in Europe trying to come to India? Because it was the most prosperous nation. Well, in 250 years we got cleaned up. We can't… well, somebody did it to us because we can't just blame them because it is also ourselves who allowed that thing to happen to ourselves. Now we are trying to build back, all right? But even though you kept them in such abject poverty, wherever these people went, they made sure not a bit of the local culture remained. Just look at North America, South America, Australia and large parts of Africa, anywhere. This one culture, after thousand years of invasions, we've still kept our culture and our spiritual process. Not for nothing. Where we are, you know, where our yoga center is, there is a police station about twelve kilometers away. This police station is supposed to service nearly a quarter million people. In different levels of economic situations, all right? Some are rich farmers, some are mediums, some are really poor. There are tribal communities, everything. There are only two… now, in the last thirty years I've been there, at a time there was only seven-eight policemen with the ASI. Today there are about fourteen to fifteen policemen with the SI. This is an upgrade that happened three years ago. Otherwise, even now with these fourteen policemen, two or three will be on leave because of something or the other. And three, four are on night duty. Hardly seven-eight policemen for a quarter million people. And these are poor people living in small hearts. People are driving in Mercedes cars up and down, but they never pounced on them, killed them and took what they have. It never happened. This is spirituality. If you've not… if you've forgotten this, this is spirituality.  Well, you don't stretch this too far. You don't take their spirituality too far. You have to settle what they need in their lives as quickly as possible. Otherwise, they won't hold for too long. But it is because of that spiritual process we as a nation have survived in spite of all the abuse that's happened to us. And don't say rural masses don't have spirituality. They are the ones who are really on. Urban masses are losing it, isn't it? But for people who are working every day, like for example, the maid in my house, for example, she… let's say she works in five, six different houses. She probably is able to make four thousand, five thousand a month. And at that point of time, she would try to… the most important thing going in her mind would be educating her children, probably feeding herself. I am saying that they do deserve to have the self-awareness and the spirituality, but how will… how will we… Okay, how will we do it? If you do not know this, seventy percent of our work is in rural India, where the poorest of the poor are, okay? It's free of cost. You walk with me in Tamil Nadu, you go into a village, the entire village will gather, not for money, not for something else. Because they know there is some tools of transformation which they can make use of. So in the city, there is money involved simply because you want it in a air conditioned room. Air conditioning is on right now. Whether I pay or you pay or somebody else pays, somebody is paying for it, isn't it? If you don't understand this, you have no sense of economy. That's the biggest problem in the country. People have no sense of economy. So they're always asking for welfare schemes. Give me this, give me that. But whether I pay or you pay, somebody is paying, isn't it? If what is paid is not accounted for, it's not going to work. This is the reason why we are poor, because we are not conducting our economics properly. Now, spiritual process is about inner strength. Especially if you are going through hardships, inner strength is most important or no? That is the time to remove it. Is it there for the rural masses? Definitely there. Much more. Seventy percent of our time we are investing. But when I come to the city, when I come to Delhi, activists come and invite me. Why are you only meeting the rich and powerful? Why are you not coming to the slum?  And I'm traveling all over. When I come to Delhi, I want to meet only the rich and powerful.  Thank you, sir.


Hi everyone, I am Gloria George from VF The Grip PLC and I was opportunity to attend the training the finance for not the finance managers training by D Brown and it's really fantastic. I must say that I am somewhat the finance literate now as I can understand the numbers on the financial statement. I love their methodology they have a lot of ways to make things simplified for example the five box analysis for the financial statement the real method as well four box everything all in one pack and it's pretty easy to understand and to apply as a non-finance manager. So yeah fantastic you should be part of this it's really lovely see you


Welcome folks, hope everyone had a great weekend out there, ready for a good June kickin' in. We have markets here, bottom line, we do have the Dow Industrial right now trading down to $211, Nasdaq is off $68, S&Ps down $23. This is the building cause, folks, for the next leg lower. That's how this is set up. Gold, gold contract down $14.20 at $18.43 an ounce. You get silver down $0.55, $21.54 an ounce, light sweet crude, $115.46 a barrel, notes and bonds. They get a lot of movement out here today, man, there's no doubt about it. You get June 1st, that's when the balance sheet's gonna start going down, they're front running it right now, there's no doubt about it. You get the 10-year, 2.848. The 10-year is trading down $26 at $119.27, 30 years off a point and a half at $140.14 and $Kingdala. $Kingdala's up $96 takes trading at $101.764, $Kingdala couldn't hold price, $Kingdala's looking for lower price. We're gonna open and take a look at the S&Ps, folks, bottom line, you get a sideways move out here and this is the building cause for lower price. We did 56 million shares out here today.  You got a high of 416, you're at 413. NDX 100, excuse me, folks. Same type of setup inside the NDX, what do you have with the NDX out here? Got to a price point of 311, high, 304 low, you got 308 million shares and the bottom line, you're gonna have light volume. Gold, gold contract, now this is really intriguing because of the fact that the matter is that you don't have gold holding price as well as the dollar. So if we take a look at this gold contract out here, you're going lower, you're down 14 and a half dollars, you got 182,000 contracts traded, there's a gap at 18, what's that one right there, right there, the lower that is, 1814, 1814's game, that's the bottom, 1840, 1841 was the lowest today, 1849, yeah, you got it, looks like 1837, still is game. And then we got good old King Dollar, we take a look at King Dollar out here and the bottom line, King Dollar failed the whole price once again, King Dollar is on its way down into this 99 area, stay right there folks, come right back.


What's going on everyone? Sam here with late in sports cards. I've been 23. Deny Immaculate, collegiate football hobby, 5 box, full case break, number 6, random team style. There's a look again at your random results. As always, you can find those on the website, under the results tab, as well as the link in the chat. PayPal just disabled my password, apparently. PayPal is bots. Hey, good luck Skywarp. What team did you get? What team did you get, buddy? Good luck, man. Hope we get you something. Thanks for breaking with us, man. Appreciate that. Vikings, okay. Here we go. Good luck. Big Jordan Addison. That'd be awesome.  That'd be awesome, dude. Patch Jordan Addison. That's Staphon Diggs for the Bills. That is Michael Meyer, Raiders, Cole Comet, Bears. Brandon L with the Bears and Raiders is Joseph D. So that'll be a random to those two teams. Going to the Saints to 65. That is going to be Kendrae Miller. Kendrae Miller. Next one to be going to the Rams. That's sick. The slogans is Hotty Toddy. Zach Evans. Ole Miss. Hotty Toddy. That's cool. Grats on that there, Rams. That's one here for Seattle, 24 at 25. That is going to be Zach Charbonnet. Congrats on that. Three colored RPA for the Seahawks. Zach Charbonnet. And going to the Falcons to 99. Bijan Robinson. Signature moves. Congrats on that. I think that's Joe. Let's go. Joe, Joe M. Joe M with the Bijan. Very nice. That is a Trevor Lawrence, 1 to 75. Hey, there is Michael. What up, buddy? Good stuff. That's Joe. Just hit the Bijan. We've got Bijan Robinson and Jamal Charles Duel. That'll be a random between the Falcons and the Chiefs. Falcons and Chiefs. He is a giant. Yep, New York Giants. We've got Rookie Otto of Deontay Banks. Deontay Banks introductions. That is going to be, I believe he's a bear or a cardinal. One of the other. Cardinals. Appreciate that, Micah. Appreciate it, man. Bijay Ojalari. That'll be going to the Cardinals. Yeah, absolutely, man. No problem at all. Appreciate that. Going to the Chargers. Max Duggan. 82 of 99. Max Duggan. Going to the Chargers. And nice one here. Rent all at the end. Quinn Ewers, 29 of 99. Introductions. Going to be a most likely top first round pick coming up in this next draft. It's like three quarterbacks that are projected first rounders in this next year. We have Caleb Williams, Quinn Ewers, then Drake May from UNC as well. Supposed to be pretty good. Currently the brother of Luke May, who used to play basketball at UNC. So that's a pretty athletic bamp. Okay, I think that is Patriots, I imagine, but he played a few places. Yup, that'll go to New England. We got Wes Welker. Nice one there. 210, 9 of 10. Yeah, he is solid. Drake May is very good. Nice. Going to the Vikings. Adrian Peterson. Game worn AP. Congrats, Skywarp. Steven on his first break here. Picking up a game worn AP from Oklahoma. That's sweet, dude. So cool. So awesome, dude. Got going to, I want to say, the Packers. Luke Musgrave. Luke Musgrave. Yes, Green Bay. Nice one there. Packers. He is a dolphin, I believe. Yup, Miami Dolphins. Devanna Shane. That is going to be 3 of 25. Going to Miami. That's on that. Dolphins. Going to the Steelers. Joey Porter Jr. Signature Moves. Redemption. Got expires in 2025. That's weird. That's so weird to think about. And 17 of 25. Two-colored RPA, Jameer Gibbs, going to Lions. We know like the funniest part about that is we're going to blink and be like, oh man, it's expired. Sorry, man. Like, geez, good lord. We're opening this in like 2027. Like, oh man, sorry. Expired two years ago. This is weird. This is really weird to say now. Going to the Saints. That's going to be a Derek Carr. That is really weird to say. This will be a random between the Jags and the Raiders. Tank, Bigsby and Bo Jackson. There's one more random. The Quinn Ewers auto is also a random to all. That'll be a random to those two teams. Got a Tank Dell. Nice one there. Going to the Texans. 32 of 99. 32 of 99. Thanks, sir. Got a Zach Evans. Nice one there. Zach Evans is a ram, I believe. Hello, Chris. What's going on, Chris? Zach Evans. Yes, Los Angeles Rams. He, I believe, is a giant, I want to say. Yes, going to the Giants. We've got an Eric Gray. 74 of 99. And then nice one here. Going to the Giants again. That is going to be Jalen Hyatt. 25 of 25. I'm just going to double check that he is in fact a giant. I believe he is. He is Jalen Hyatt. He is too pure degenerate for this chat. He works here. He works here, Chris. So trust me, we know he is pure degenerate. Jason A with the last tiered spot. Jason also with four spots in the mixer. Jason G with a spot in the mixer. Thank you, Jason. Selling us out on tiered and we're down to just 15 to go. 15 to go. I don't know where those came from, dude. Thank you all. 15 to go on that mixer. Crying inside is a giant spin. Dude, that Gameworn AP is pretty sweet, though. I mean, Gameworn stuff and Panini, let alone like draft pick stuff is very hard to come by. So that AP is sweet, dude. AP's still got a great market, too. My list. Ten megatron. I'm going to get that. That I would throw away. That is bowl season. Justin Fields, short print going to the Bears. Got to A. Xavier Hutchinson. He's a Hexan, I believe. I have not seen any Hainer so far. Yup, Euston Saxons. Got Rookie Autumn, Elite Cunningham. That'll be going to the Patriots, 4 of 99. You have not seen any J. Kainer yet. I'm assuming he's going to be in here, though. We've got a Steelert, Darnell Washington. 23 of 99. Aha, I remembered. 36 of 99. Josh Downs going to Colts. I got it. Aha, 36 of 99. Josh Downs. And we've got Marvin Mims going to the Denver Broncos. 63 of 99. Very nice. Mr. Marvin Mims. Aha, see, there you go. Told you S2. All right, we've got like four randoms to do or something. See how we're going to be doing a random for the Michael Meyer, Cole Comet, the Bijon, Jamal Charles, the Quinn Ewers to everybody, and then Tank Bigsby and Bo Jackson. Let me get these all set up real quick. Need an additional random window. Got the Notre Dame. Tens. Michael Meyer's Raiders. Raiders. There's got the Texas RVs. Falcons. Falcons. Auburn, right back. Tank Bigsby's Jaguars. Going to do these randoms real quick, everybody. Going to start off with the Notre Dame tight ends. That's Michael Meyer of the Raiders and Cole Comet of the Bears. Roll the dice seven times. Whatever team's on top, we'll get that card. It'll be one, two, three, four, five, six, seven. Going to the Raiders. Raiders on that one. Going to do the Texas running backs. Bijon Robinson of the Falcons. Jamal Charles of the Chiefs. Roll the dice on that one. Going to go 11 times on that. One, two, three, four, five, six, seven, eight, nine, 10, and 11. Going to the Chiefs. Chiefs going to get that one. We've got the Auburn running backs. Tank Bigsby of the Jags. Bo Jackson of the Raiders. Whatever team is on top after 10 times. Look at that one. One, two, three, four, five, six, seven, eight, nine, 10. Going to the Raiders again. And then we've got the Quinn Ewers auto. Random to everybody. Eight times on that. One, two, three, four, five, six, seven, and eight. Going to the Jags. Going to the Jags. All right, guys, let's do a quick recap. Add the dual jerseys, dual jersey, dual jersey. That is Xavier Hutchinson, Adrian Peterson, Quinn Ewers, Marvin Menzotto, Darnell Washington, Josh Downs, Malik Cunningham, Jaylen Hyatt, Eric Gray, Zach Evans, Tank Dell, Jameer Gibbs, Joey Porter Jr., Devana Shane, Luke Musgrave, Max Duggan, BJ Ojalari, Deontay Banks, Zach Charbonnet, Zach Evans, Kendra Miller, and Bijan Robinson. That's going to do it for the break, guys. Thanks again for the film. Up next is number seven.


Good afternoon. I remind members that Covid-related measures are in place and that face covering should be worn when moving around the chamber and the Holyrood campus. The next item of business is portfolio questions. A portfolio is education and skills. If any member wishes to ask a supplementary question, the members should press their request to speak buttons. I will put an R in the chat function if they are joining us online during the relevant question. 1. Alex Cole-Hamill Thank you very much, Deputy Presiding Officer. Do you ask the Scottish Government how many unemployed teachers there currently are in Scotland? We are firmly off the view that we will need all possible teaching resources at our disposal to compensate for any loss of learning suffered since the start of the pandemic, as well as to bring much-needed resilience to the education system at its challenging time. Since the start of the pandemic, we have provided £240 million of additional investment specifically for the recruitment of more education staff. In addition to this funding, we are providing local authorities with permanent funding of £145.5 million per year to support the recruitment of extra teachers on permanent contracts. I am very grateful to the cabinet secretary for that reply, but I have to tell her that this is still a major problem. Over the last 24 hours, I have had a flood of complaints from newly-qualified teachers. One was so desperate that she was spending £400 a month travelling from Edinburgh to Glasgow for a teaching job. Another had retrained to become a teacher, but is now back with her former employer. One said, I have had enough. My mental health has been affected. My life is on hold until I get a permanent post, but there are none for any of us. The education secretary's complacency is crushing the careers of newly-qualified teachers. When is she going to get this sorted? As the member knows, the responsibility of the recruitment of staff is directly with local authorities and does not sit with national government. National government's responsibility is around funding. I am detailed in the answer to Alex Cole-Hamilton, particularly around the base lining of funding that we have provided to local authorities to ensure that local authorities can recruit extra teachers on permanent contracts. Teachers published show that teacher numbers are increasing for the fifth year in a row. There are now more teachers than at any time since 2008. The ratio of pupils to teachers is at its lowest since 2010. We are undertaking firm action on that. I have demonstrated that in my original answer. We will continue to do so to support our valued teachers at this time. As I said, we know that their skills and expertise are required to support our young people during this difficult time. There are a number of supplementaries. I hope to get all of them in so brief questions and answers if possible. I will entwide first. In a similar vein, to ask the Scottish Government for an update in progress on recruiting more teachers to Scotland schools. The additional £240 million that I mentioned in my original answer to Alex Cole-Hamilton has already been provided and has supported the recruitment of more than 2,200 additional teachers and more than 500 support staff. The permanent funding of £145.5 million per year that we are providing to baseline local authority funds will allow the recruitment of even more additional teachers on permanent contracts. As I said in my answer to Alex Cole-Hamilton, that means that we have more teachers than at any time since 2008. Given the number of teachers looking for employment and the additional roles being recreated, does the cabinet secretary share my concern that Dumfries and Galloway council are telling parents, particularly in smaller rural schools, that they may have to cut back on teacher numbers and strictly enforce ratios? Every council will receive additional funding from the Scottish Government, as I have laid now in my answers to two members. That would include Dumfries and Galloway council. I would encourage the council to ensure that that additional funding that has been baselined into a settlement is used to ensure that permanent staff are recruited. I am very grateful, Deputy Presiding Officer. Excuse me. I thank the minister for confirming that the 1,000 new teachers and the 500 people support assistants that were announced on 2 August have indeed been employed. Can she confirm, out of that number, how many are qualified to work with ASN children? As the member will know, there are children who all teachers will work with that will have additional support needs. There are particular arrangements and the determination that this Government has to ensure that we are supporting children with additional support needs further in our work with the Colleagues in the Scottish Green Party. I will certainly keep Mr Whitfield up-to-date on the work that we are undertaking on that. Willie Rennie is truly astonishing that the education secretary, after months of asking, still does not know how many unemployed teachers there are in this country. She is the education secretary and she should know the answer to that. She cannot simply pass the buck to councils. When will she get a grip of this situation and improve the careers of teachers, rather than crushing them? I can only go through some of the figures that I have just detailed about the action that has been taken since I became education secretary, including the baselining of the funding to ensure that we have permanent contracts that are available. That, of course, is a matter for local authorities, but local government has been provided the funding for those additional posts through the national government's work. We are determined to continue that work. Of course, we have a commitment during this parment for the employment of at least 3,500 additional teachers. That is over and above the additional 1,400 teachers that were recruited during the pandemic. This Government has taken action and will continue to take action to support our teachers and young people. To ask the Scottish Government for an update on the Discover Cyber Skills programme. The Discover Cyber Skills programme was delivered by Skills Development Scotland between September 2017 and March 2021. The purpose of the programme was to build young people's, parents and teachers' understanding of good cyber practice and also to inspire young people into cyber security careers. Over the four years of the programme, SDS engaged with over 130,000 pupils in schools from every local authority in Scotland. In addition, the programme provided upskilling to teachers through the development of a toolkit to support the delivery of cyber skills sessions to young people. The biggest show that there has been a 29 per cent decrease in pupils taking computing sciences at a higher level between 2016 and 2020, as well as a significant decline in computing teachers over the past decade. What action is the Scottish Government taking to increase uptake in this area to ensure that we have a pipeline of talent in place to meet serious challenges such as cybercrime, which has nearly doubled in the past year? I concur with the direction that the member is taking with her question to say that this is a very important issue. We need to ensure that we have the young people with the adequate skills in this area. Part of that is, of course, around teachers that are qualified in this area. The latest data shows that there was an increase in the number of teachers that have computing when compared to what was there in 2019. We know that there is, of course, more work to do on that issue, but that is exactly why we are taking for work, for example, from the Logan review to ensure that we are moving forward with this agenda. Is it a very pertinent question? I have regular representations from businesses in Dundee who tell me that they could create hundreds of jobs in the city where the sufficient skills are available locally and the labour pool does not exist. I commend the work of local universities such as Abertau, but, unfortunately, they cannot use computing as a compulsory subject because so few state schools now offer higher computing in their schools. What concrete action can be taken immediately to address the deficit that is being identified and to make sure that we can get people who, instead of in the first year of their courses, have not relearned the subject that they should have learned at school? I mentioned in my original answer, or perhaps my simple, mentory answer, the work that was going on following the Logan review. There were a number of pieces of that that were to do with schools. I, of course, recently met with Mark Logan and Senior Fitzhull to discuss that work. We are due to have further meetings in the next few weeks about those proposals and how we can take that forward. There are a number of areas of action that have already been undertaken by Education Scotland and other agencies in this area, but I recognise that we need to do more on that. That is exactly why I have continued engagement with Mark Logan and others interested in this area. Both Skills Development Scotland and the General Teaching Council have previously noted the value of bringing computer experts into the classroom to upskill both teachers and pupils on issues such as cyber security. Can the cabinet secretary advise whether the Scottish Government has any plans to bring further industry experts into school on other broader tech-related subjects? One valuable form of expertise is from industry voices who can work with schools to inspire young people and to support the teacher. There are a number of partners already working with schools to introduce industry skills and to support this work. Of course, Skills Development Scotland is heavily involved in establishing a framework that brings agencies together to help them to tackle that challenge. It is an important area of work that we are keen to do more on to ensure that schools have close relationships and discussions with industry and with local and national businesses, and that Skills Development Scotland is very heavily involved in. To ask the Scottish Government whether it will provide in the update on its plans to incorporate the UN Convention on the Rights of the Child into Scots law. We remain committed to the incorporation of the UNCRC to the maximum extent possible as soon as practicable. We are considering the implications of the Supreme Court judgment and how best to take forward incorporation. We are undertaking targeted engagement with stakeholders and options before final decisions are made. Options also need careful consideration to ensure that the address areas found to be outwith competence deliver the UNCRC policy and avoid further challenge. The Deputy First Minister will come back to Parliament in due course with her proposals. Although the Supreme Court judgment means that the UNCRC bill cannot receive royal assent in its current form, the majority of the work in relation to the implementation can and is continuing. I thank the minister for that answer. This issue is too important for there to be further delays. Every day that passes is another day where those rights are not enshrined in law. I know that the Scottish Youth Parliament is understandably concerned about getting those rights in place. We have also issues within Scotland such as our national call for vacancy agency being subject to statutory action from the EHRC, which shows how vital those rights are. When can this Parliament expect a timetable for our revised bill and how soon is it in due course? It is fair to say that the Scottish Government is bitterly disappointed that the bill has been delayed, but we remain committed to the incorporation of UNCRC to the maximum extent possible and as soon as practicable. We are committed to a three-year UNCRC implementation programme in collaboration with public authorities, children and young people, during which we are investing £4 million a year to support a fundamental shift in how children's rights are respected, protected and fulfilled in Scotland. The work is progressing. As I said in my original answer, the Deputy First Minister will return to Parliament with her proposals. I have got a number of supplementaries. I would hope to get them all in, but brief supplementaries. Please, David Torrance, first. Thank you, Presiding Officer. Does the minister agree that the straightforward and quickest way of incorporating the United Nations Convention on the Rights of a Child would have been for a bill that Parliament unanimously voted for, receiving royal assent without objection from UK Government law officers? Yes, I do. I agree with that, Mr Torrance. The Supreme Court judgment makes it plain that we are constitutionally prohibited from enacting legislation that the Scottish Parliament unanimously decided was needed to enshrine and fully protect the rights of our children. This Government believes that the children of Scotland deserve to have the commitments made to them fulfilled, and our determination to do that is undiminished. It is with urgency that we are carefully considering the implications of the judgment and how best to take forward this important legislation. It has been over a month since the Scottish Government was told that its bill went beyond the powers available to the Scottish Parliament. Tax payers money has been spent so that SNP can play cynical constitutional games with children's rights. Therefore, can I ask the minister whether she believes that using almost £200,000 of tax payers money to further their national agenda is appropriate use of public funds? Can we listen to the response, please, all sides? Perhaps the member being a new member is not aware that her party also voted for that legislation. It was voted for unanimously by this Parliament. I have asked for a bit of quiet so that we can hear the response. It is very difficult when you have the opposing benches shouting over you. The Parliament is committed to ensuring that the children of Scotland's rights are upheld and respected. We will continue to work to make sure that that happens. Both front benches were engaged in shouting over others, so I ask for a bit of courtesy for speakers. I am going to have a final, brief supplementary answer called Hamilton. The Government talks about moving with urgency and being bitterly disappointed, yet it could foresee that that would have happened when whitehall officials in the process of the bill had informed Government officials that there were problems. We were never told on opposition benches to the left or right of those problems during the transit of the bill. They could have easily remedied at that stage. Will the Government now take this opportunity to explain to the children of Scotland why they withheld that information? I am not quite sure what point Mr Cole-Hamilton is making there, as he seems to be asking several questions of which I am not quite sure what he was actually wanting me to respond to. The duty of this Government is to ensure that children of Scotland have their rights respected. We will continue to work at pace to ensure that the will of this Parliament is upheld. Question 5, Liz Smith. To ask the Scottish Government what measures are in place to protect the autonomy of Scotland's universities. The autonomy of Scotland's universities is protected through their legal status as independent charitable bodies. The higher education government Scotland Act 2016 also expanded the statutory definition of academic freedom, requiring our universities to uphold the academic freedom of persons engaged in teaching or the provision of learning or research at the university. Liz Smith. I thank the minister for that reply. I am sure that he agrees that both university autonomy, which includes the academic freedom that he has just referred to, has been part of the success of the university system for hundreds of years precisely because it protects institutional diversity and specialisation. The system reflects the likes of the Royal Conservatoire Rural College and Glasgow School of Art. Does the minister accept that recent developments were to free speech in the university system, both at UK and Scottish level, threatened to undermine the concepts of autonomy and academic freedom? Will he explain to Parliament what action the Scottish Government is taking to ensure that university autonomy is never diminished? I understand why the question is being asked. In my experience, I have not had a single institution contact me or a single academic contact me expressing concerns about what the system here in Scotland is operating to. Our responsibility is to ensure that we enshrine, protect the autonomy of institutions and, indeed, academics. That is enshrined in the Higher Education Government Scotland Act 2016. We expect institutions to adhere to that. Of course, the definition sets out the academic freedom that includes the freedom within the law to hold and express opinions, question and test established ideas or to see wisdom develop and advance new ideas or invent proposals or present controversial or unpopular points of view. Our university should be able to do that. I see nothing that incumbers them from being able to do that. Question 6, Pam Gosall. To ask the Scottish Government what action it has taken to ensure that the education curriculum provides pupils with sufficient skills to build a digital economy. Cabinet Secretary, curriculum for excellence, experiences and outcomes for the technologies have been reviewed to ensure that all young people are learning up-to-date computing and digital skills in both primary and secondary schools. In addition, the Government has committed to fully implementing the Scottish Technology Ecosystem Review carried out last year by Mark Logan. Part of that review is the Scottish Teacher's Advancing Computing Science initiative, which was announced in our programme for government. That will run for and by computing science teachers helping them to share best practice in computing science across all schools, promoting digital sector career opportunities and developing innovative teaching materials to be used for extracurricular computing clubs. I thank the cabinet secretary for that answer. Official figures have revealed that there were 766 computing science teachers responsible for 25,000 pupils in Scotland in 2008, but by 2020 there were 595 teachers educating fewer than 10,000 pupils and less than 2,000 of those were female. What action has been taken to address the gender divide in the computer science studies? Some of the discussions that we have had in relation to a previous question touched on the work that the Logan review has proposed. As I said in my answer to that, there is already a great deal of action being undertaken by Education Scotland and other agencies. That includes our work to increase capacity for initial teacher education for computing science, support for coding clubs and work to make it easier for schools to work with industry and bring industry voices into schools, as I also mentioned earlier. In response to the Logan review, we are taking forward the STAC's proposal to establish that teacher-led group. In relation to the gender issue, which Ms Gozo rightly points out, there is a concern around that in all stem areas. It is something that is in the Scottish Government's stem education strategy that was produced. It is something that we are determined to take further action on, as we did in that strategy, to ensure that we can deal with that gender divide. The current Secretary of State for Education, who has added investment in new facilities, can also be a catalyst to improving both interest and skillset with regard to the digital economy, which therefore makes the case for a new-build West College Scotland campus in Greenock, even more stronger. I agree that good quality learning opportunities that develop digital skills for our young people are vital at all stages in the education journey, including our colleges. I understand the minister for higher and further education visited the Greenock campus of West College Scotland with Mr McMillan in September. I heard first hand about the benefits that a new-build campus could bring. Of course, he will be aware that the Scottish Funding Council has been working on a medium-term college estate strategy, which will help to set priorities for investment should funding become available in the college sector, and we will work closely with the funding council in taking that forward. Very grateful, Deputy Presiding Officer. Could the cabinet secretary tell us how many high schools are offering higher computing this year? If the member would be agreeable, I will get to him in writing about the particulars on this issue. I am not entirely sure that that exact figure is in my briefing. Question 7 has been withdrawn. Question 8. Christine Grahame. We have some assistance from Ms Grahame. Expectation levels have risen to an almost unsustainable level. Then I am doomed to fail, your deputy, Presiding Officer. To ask the Scottish Government what improvements it considers it has made in the delivery of education in Midlothian South, Tweeddale and Lauderdale. Schools and early learning and childcare settings across Midlothian South, Tweeddale and Lauderdale have benefited from a range of national and targeted initiatives to support the improved delivery of education. 1140 hours of high quality funded early learning and childcare, pupil equity funding and the £500 million of Covid funding to support education recovery and wellbeing when it will improve outcomes for children at a local level at our £2 billion learning estate. Investment programme is funding a number of projects in Midlothian and the Scottish Borders, including a replacement at Beaslack the Community High School, Penicook High School, Gallowshills Academy and Peebles High School. I thank the cabinet secretary for a detailed answer. Following the admission after four years by Scottish Borders Council of liability for its failings in relation to some of my constituents' children attending Tweeddale's support unit, currently subject to an independent inquiry to which I shall be giving evidence, what improvements specifically has the Scottish Government made to support children with additional support needs and their families in my constituency and elsewhere in Scotland? My thoughts are very much with the children and the families that have been affected by any issue of concern in any part of our education system. All children, of course, have the right to be cared for and protected from harm and to grow up in a safe environment in which their rights are respected and their needs met. I do note that Scottish Borders Council's announcement of an independent inquiry would be inappropriate for me to comment further at this stage, but I recognise Ms Grahame's continued involvement and concern around the issue. I give her my reassurance. I will keep a close eye on the proceedings on this and I would be happy to discuss those issues with her in due course, should that be more appropriate once that inquiry has taken place. Thank you, cabinet secretary. That concludes portfolio questions and there will be a slight pause as the front bench is changing.


Thank you very much. Yeah, so also welcome from my side to the bite-sized talk and after Harshal introduced us last week to modules and DSL2 in general. I want to try and give a brief overview about how you can now make use of the modules in a pipeline. Sorry. Okay, so to get started, I just want to briefly recap what a module is. So it's an atomic process that cannot be reduced any further, and it usually contains like a single software tool like Fast2C, for example, and it can be used within a pipeline and also shared between different pipelines. And to make use of the sharing, there's the NFCore modules repository on GitHub, where you can find many of these modules already. So to make use of this modules repository, there's a new or already released, but there's an existing sub-command in the NFCore tools. Just a brief recap, you can install it with Kipp and Condor. And if you then run NFCore modules, you get a list of sub-commands that you can use to interact with this repository. And if you then run one of the sub-commands, with minus, minus help, you get some instructions on how to use them. So I just want to, over the next couple of slides, briefly introduce you to some of the sub-commands that could be helpful for using modules in the pipeline. So I think one of the first thing you may want to do is try to find a module that you could use, and you can use the NFCore modules list for that, and it will just print out all the modules currently available. And they are subdivided in tool and subtool because many tools like SAM tools or BCF tools have these sub-commands that are then their own module, basically. And then other tools like FastQC that don't have any sub-commands, they are just a tool. Okay, so if you look through the list and you find a module that you like to install, you can run NFCore modules install, and then your pipeline directory and the tool name, and that will install this entire module for you without you having to do anything else. And it then looks like this. So on the left hand side, there is a completely fresh new pipeline that I created with the template, and then around NFCore modules install, and then on the green, you can see where this module ends up. So the modules then basically creates, there's a sub-folder called NFCore software, and there you can find the FastQC module and with three files, the functions and F, the main and F, and the meta.yaml. In the meta.yaml, you can find documentation like what type of input and output does this module take, who wrote it, what does it do, these sort of things. In the main NF, that's where the extra magic happens, where FastQC is run, and then in the functions and F, there are some helper functions that are needed. All right, so one of the, I guess the opposing step to that is how to remove a module that you don't want to use anymore or that you erroneously installed, and for that you run NFCore modules remove, and that removes this entire sub-folder that had FastQC with this function's main and meta file in it, and it's gone. And then you can, you have nothing, you don't have to do anything else there. All right, then sometimes the module will get updated, maybe the software was updated and this was propagated already to the module, and you would like to use this newest update. So how to make sure, first of all, that you have the newest version, and then how to update it. So to check this module, you can run the NFCore modules lint on the directory to check our modules, or on a specific module like FastQC, and it will check, among others, whether or not you have any, missed any changes. And then to update it, you currently have to remove it and then reinstall it, but in the future, Kevin is already working on creating an update sub-command that you can then use. Okay, and then last but not least, what do you do if the software you're looking for is not an NFCore modules, and you basically have two options. One is to add it to NFCore modules, and the other one is to create a local module. So to add it to NFCore modules, this is usually really helpful if some other people or some other pipelines will use the software as well. And if you're unsure about it, you can always check the issues. Maybe somebody has already started working on it or requested it, or you can ask in the module select channel, and Kevin will talk about how to actually add NFCore modules in his talk in a couple of minutes. And the other option is to create a local module. So this is useful for software that is really specific to your pipeline. And you can run NFCore modules create, which will create a local module for you. If you look in this box, you can see the NFCore modules folder that we already saw and we've seen before, and the local sub-folder in which your tools will then live on your process will live. And a lot of the things that Harshal and Kevin covered in recovery in both of their talks is relevant here, even if Kevin's talk, for example, is more tailored towards NFCore modules, but a lot of the functionality will be similar there. Okay, so this NFCore modules create gives you this file here with many to-do statements and little help messages. And then you can start filling it out and try to get your tool to run here and hopefully the to-do statements help you figure out what exactly you will need to do in each and every step. All right, so now that we have modules either local or NFCore modules, we want to start writing actual workflows and pipelines. And they are two different types of sub-workflows. So these are chains of multiple modules with some sort of higher-level functionality, like all the QC tools that will be run on FastQ files, and then actual workflows, which are end-to-end pipelines, and then DSL1. We've known them as like these large monolithic scripts. But in DSL2, this is the combinations of modules and sub-workflows. And this is really taking one input and then producing a final output. On the right-hand side, I visualized the file structure a little bit. This is the current file structure of the Viral Recon pipeline. And for modules we have local and NFCore ones. I showed you those before. And then for sub-workflows, it will be a similar structure. Just some sub-workflows will be relevant to many pipelines, such as QC. And then we have the workflows. And in this case, I think Harsha separated by the input types of data here. So for Illumina and Anupor data, there are different types of workflows, which will then be called from the main NF. And these workflows are consistent of sub-workflows than modules. Okay, so to use a module, you have to install the module or create a local one. And then the next step is to adapt the modules.config. And this is really where the sharing of tools on software makes it easy, because here we can actually share the same software, but we can specify for our specific pipeline which parameter should be set. So for mark duplicates here, it says exactly which parameter should be used. And there are a couple of different ones. I highlighted here the arcs one to give you an example. And then some modules you will see have an arcs two line. And this is because, for example, for mapping immediately, SAM tools is run afterwards within the same module. So with one with arcs, you can specify it for BWM, for example, and then with arcs two for SAM tools. Okay, then the next step, if this is only has to be done once, you have to include this modules.config in the next for config, and then at last include the module into a sub-workflow or into a workflow. This looks like this. In pink, you can see the include statement you include, this module BWMM, from the path where it lives, so modules and of course software and so on. And then you add these options, these params.BWMM options that were previously specified in the config, in the modules.config, so this is how you can pass them down. And then the sub-workflow that we have here, we have three scopes. So take main and emit, and then take, it specifies the input data as channels. So here we have reads and indexes or indices for my little test workflow. In the main, this is where the modules come to work. So we have the BWMM module that was included up top and the SAMtoolSort module that was included. We can run BWMM and then take even the BWMM output and directly run it into a SAMtool, put it into a SAMtoolSort. And last but not least, from the sub-workflows, you can emit named outputs to then do other things and other sub-workflows or modules. So here we can now name our output from SAMtoolSort out BAM and name it, I don't know, sorted BAM. And then up in the workflow, we can access this sorted BAM directly to do something with it. Okay, so last but not least, we have to combine the sub-workflows to workflows or sub-workflows or modules to workflows. So here we, in the header, we have to include statements for the workflows or the sub-workflows in the FASTQC. So from once from the modules and of course software and one from sub-workflows local. And once again, we have to add the parameters to pass them down. And I highlighted here again in yellow for you to be able to track this a little bit. And for the FASTQC module, we just need to specify the parameter for FASTQC. So modules and then in the brackets FASTQC. And but for the sub-workflow, you have to specify all options for all the tools in there that you would want to specify. So here we have BWM and SAMtoolSort. And this is exactly now this field that was originally specified in the modules.config. Okay, and then in the workflow and F here, I've just run the module FASTQC on my input reads that I got and then the sub-workflow on also the reads in the index. I get the sorted BAMs output. And then I can do some more steps with that. Okay, so and then I want to show you how it looks like to add the workflows to the main NF. And here I took the ViraRecon one that I mentioned earlier. So here you can see now that we have three different workflows that are actually possible. And they are apparently dependent on the input data. And the whole main NF now becomes really lean though. So there's a bit of header still here, but overall the entire main NF is less than 100 lines currently. And it makes it really easy to now track for your input data, which workflow is run. And then you only need to look at the SRA download workflow to really see what's happening. Okay, and then two more things I want to mention that are important. So you always will have to adapt the multi QC module somehow to customize it for your tools. So you will have this is similar to the DSI1 version, but you can collect all the metrics in the workflow script, and then pass it down to your multi QC module on the right hand side. You can see all the different input data. So this is one where you have to create a local module for and you can then collect this data from the, I don't know, fast QC module, for example, here.  So each module must emit a software version. Just because we need to track all the two versions, and you can collect them all by creating an empty channel and then mixing these versions in from the sub workflows, you can propagate them up by these named emit versions. So for BWM version, you can get the version from the module and then access it again in your workflow script as test sub workflow out BWM version, then run your local module in the workflow. All right. And with that, I'm already in the end. Don't forget to join us for the next hackathon. And if you have questions, we will have a Q&A session in the end, or ask them in bite size later on modules.