Welcome to MOOC course on Introduction to Proteogenomics. Our next speaker is Dr. Karsten Krug who will talk about effect of mutations on signaling pathways and how they could be studied using softwares like MIMP and active driver BD. He will talk about the frequency of phosphorylation and factors which may lead to specific kinetic activity. He will also talk about tools like Motif X and phosphosite plus for sequence motif analysis and also calculate the frequency of most recurring amino acids near the site of phosphorylation. So let us now welcome Dr. Karsten Krug to talk in more detail about the role of various mutations on signaling pathways and also to tell us about various factors which may help us in understanding how phosphorylation can be understood in a biological system. So I will first going to give you like a short motivation. So why we want to do that and then I will be very specific and I will talk about two specific software tools that tried to you know that tried to study the impact of mutations on phosphorylation networks. So as being just presented so there is millions of single nucleotide variants known in the human genome and many of them are associated with certain human diseases but for most of them we do not know the exact molecular mechanism that you know causes this genotype to phenotype association and as we have learned yesterday during Kelly's and David's talk and also during the hands-on so mutations can if they are located in protein protein regions they can be non-synonymous meaning they can lead to a single amino acid substitution so they can change an amino acid in the protein sequence. And here so I mean there is many different synonyms for these events a non-synonymous SMV is single amino acid variants or single amino acid variants. So these are all you know referred to the same kind of event but you will find all of these in literature and of course many of these single nucleotide or non-synonymous single nucleotide variants they affect sites or amino acids that can be post-translationally modified like phospholation acetylation or ubiquitination and actually these modifiable amino acids occur very frequently in the human genome. So what you are looking at here are the frequencies of all 20 amino acids in the human proteome. So the most frequent amino acid that occurs is losing but then on the second place we already find serine which can be phosphorylated right. What is what are highlighted here are serine's license theonine's and tyrosine's license are you know can be modified by acetylation or ubiquitination. So these are the most well studied and you know so we have the technology to study these modifications on a large scale. So that is why I highlighted those amino acids here and if you just look at the like you know the overall frequencies of these four amino acids they make up for 22 percent of all amino acids that occur in the human proteome. So it is very likely that the mutation affects these modification sites and we are asking the question what kind of consequences does that imply in downstream signalling events and there has been many studies out there that tried to you know decipher these kind of relationships and I am just highlighting a couple of those here. So you know the most basic kind of approach to to take here is to look at kinase sequence recognition motives. So kinase phosphorylates its substrate and one mechanism to ensure that the kinase you know specifically identifies its substrate is you know is given by very local interactions meaning it is a it is the amino acid sequence the properties of the amino acid sequence around the phosphorylation site. So basically as a probably many of you have seen these kinds of sequence logo motifs here where in the center you you are looking at the actual modification site. So this overall kinase B mostly phosphorylates serines but also threonines and if you look at the frequency around it substrates you know blanking the sequence you see that there is a strong enrichment of an arginine at position minus 2 we got you know relative to the phosphorylation site. So this is this is what we call the sequence recognition motif of overall kinase B. So basically it recognizes the arginine at minus 2. Any questions for that? Well I mean overall kinase I mean there is two classes of kinases. So one separate class is tyrosine kinases which only specifically phosphorylated tyrosines. If you look at all known substrate of overall kinase you find most of them have a I mean most sites are serine sites and then there is a small fraction of threonine sites. So overall kinase cannot phosphorylate tyrosines. Well so we are looking so this. So the zero position is actually the site where the phosphorylation happens right and we are looking around this phosphorylation left and right. Okay so and there is again there are many tools that can generate these kind of logos and perform these enrichment tools and you know the common principle of these tools is to test for enrichment of amino acid patterns that surround this phosphorylation site and you compare it against some background data set. So for example you have your phosphorylation site data set that you've acquired in your lab you know you have like in this case maybe like 20 sites or so. So these can be all of your all detected sites in your experiment or if you let's say you specifically inhibited the kinase and now you are looking for all phosphorylation sites that are down regulated upon inhibition. So this could tell you these are very likely substrates either direct or indirect of this particular kinase. And then you compare so you compare the frequencies that you obtained here against the background data set and this can again be all detected phosphorylates in your data set or you could use all known phosphorylation sites in your human proteome for example. So as I mentioned there are several tools so two very popular ones are motive acts and the sequence motive analysis tool on phosphocyte plus. So motive acts was probably one of the first if not the first tool which was published back in 2005 by Steve Gignes lab at Harvard Medical School. And so the basic principle again so you have your phosphorylation sites so the center is where the phosphorylation happened and then we are looking at the surrounding amino acids. And from that you can very easily build up this kind of frequency matrix where in the columns you have your offset positions again in the like zero means these are my this is the actual phosphorylation side and then you're looking seven amino acids to the left and seven amino acids to the right in this case again this is very arbitrary so some tools use different you know sequence windows length and so on. And on the y-axis you're looking at the like all 20 amino acids and then you just basically account to frequencies of these amino acids in your data for example in this case we would have like three glycins exactly we would have two prolings and and so on and so forth right. So it's very easy to to build up this kind of frequency matrix and this is exactly what motive x stars so it builds up these two matrices one is derived from your actual phosphorylation of interest and the other matrix is derived from your background data set and from that you can then calculate a binomial matrix binomial probability matrix where you for each position in your sequence window you ask and for each amino acid you ask the question for example in this case how many times do you observe a proline at position plus one in your data set this you can calculate for each amino acid and each position and you compare it against the background frequency which should derive from your background data set which can again be the the entire human podium or your or all of your detected sites and from that you can then calculate or generate the sequence windows you just put up an example which is based on the seven sequence windows that that we've looked at a couple of slides earlier so if you actually calculate this probability to observe k out of n so n in this case was seven so we looked at seven sequence windows we observed four prolings at position plus one you know and the background probability of a proline in the human podium is roughly 0.062 and you know in r you can just feed it into into the binomial test function and you get a p-value that this indeed although it's very small sample size it would be statistically significant so four out of seven so and so forth this again you would do for all amino acids in all locations so but you don't have to go too much into detail here's a motive x dusted like in an iterative manner so it it first of all takes all sites that you feed in and extracts the most significant sequence motives from that set and then it removes those from the initial set and repeats the analysis so that's one you know specific property of the software okay so now we know how we can look or how we can determine these specific kind of sequence recognition motives but now what happens if these if a mutation happens in this kind of region around the phosphorylation side so there's actually three different scenarios that can happen so one is directed so you actually so this is the wild type here this is the mutated version so in the wild type you have a serine which is actually phosphorylated and this serine now due to a mutation gets mutated into a histidine so it cannot be phosphorylated anymore and of course this can also happen the other way around right so the histidine can be mutated into a serine which now maybe present a new phosphorylation side which can be recognized by kinase so it can be either like you can lead to a genesis or phosphorylation side or to a destruction of phosphorylation side so the other possibility that could happen so it does not happen at the exact side but it can happen very close to the phosphorylation side so in this case we have this poline here at plus one which now is mutated into an arginine and the kinase that was able to recognize this poline can now not phosphorylate this specific serine anymore because the poline is gone right so meaning in this case we would lose this phosphorylation side again it can also go the other way around or it can also happen that you just change the sequence kinase motif so in this case in the wild type it was kinase A who recognized this motif and now due to a mutation this motif changed into another motif that this can be recognized by kinase B so these are the most simplest examples of these kinds of events that we are looking at. There is also like further or like events that happen you know further apart from the phosphorylation side for example if a mutation hits a kinase domain which contains the catalytic function of this kinase it can also change the it can lead to to a burned kinase activity so here on this slide I just presented a couple of tools so there's many tools out there already and you know people have started looking into that 10 years back already but also recently there are lots of new developments so one so these are actually the two tools we are going to have a closer look at so one tool is called MIMP and the other is called active drive it will be just want to highlight so this other tool here G2P database so genome to phosphor database which has been developed in David Fanyu's lab as well yeah. But the there it was asking will your side be the phosphor database like in the first time might be that the sitting position is the full position yes but in my sequence it was sitting 32 so I was not able to use the promoting text because I am biologist I am not a programmer yes so is anyone of these tools as you know all of these tools is anyone of these tools solved by problem? No well these tools don't solve your problem there are so this is one step actually upstream of this type of analysis so many of these tools that actually take the wrong mass spectrometry data and do the database search and create result reports on a phosphor side level let's say so many of these tools already have these kind of sequence windows in their result table so this is actually not a peptide sequence right it's the sequence window which is always the same length so in this case it's always 15 amino acids it's this and the modification side is in the center so this is something that many tools create like Maxquon does it like Spectromil does it I'm not sure whether Podium Discover does it as well yes yes I see your problem so what you would have to do you need to hire a programmer that just takes the data takes the database and creates a sequence window so well you just use another software so I can highly recommend Maxquon okay so MIMP does exactly what we've just talked about so it predicts the impact of a non-synonymous as in we is on kind of substrate interactions so it predicts kind of binding affinities and how or whether a mutation rewires Podium or like phosphosignal networks and it compares so that's basically like the principle here so it compares the effect of mutated and valve type samples and one specific property of this tool is that it uses a Bayesian approach to construct like these in this case position weight matrices which is probably which is very similar to these amino acid frequency matrix that we just looked at so this has been published in HM method like three years back and there's an online tool which you can just use but there's also like an R implementation of that package and this is actually what we are trying to use in the hands-on sessions so I hope that that we will be able to do that so we'll see so the key features is at first builds kinase binding models using known substrate sites so very highly curated very well known phospholation sites that have been determined to be a substrate of this particular kinase it used this information to build binding models and these models are already included in the software right so this is nothing that you have to worry about so then calculates for a given phospho sequence that might be now coming in from your data set it calculates the kinase binding score for each kinase it calculates the score how likely it is that this phosphoside has been phosphorylated by this specific kinase and again so you can upload your own phospho sequences or you can just query all phosphoside sequences in phosphoside plus does everybody know about this this resource here phosphoside plus so I can again highly recommend to to check that out so at the end of my slides there are like all references that I am going through here are included so you can just go through the papers and check them out yes so the first part is calculating kinase binding specificities and then it puts in mutation data which again you can upload your own mutations or it take or you can specifically look for TCGA mutations and then it does its prediction so today in conclusion I hope you have learned that kinase activity gets affected due to the amino acids which are present in the surrounding of the phosphorylation site hence if we know the correlation of amino acid to phosphorylation specificity then we may change the expression pattern of a gene we also heard that motive X follows an iterative workflow which provides us reliable and confident amino acid sequence which could result for the phosphorylation regulation mutations can lead to genesis modification or destruction of p sites resulting in iodide pathways Dr. Krug also interested many tools which can be used for correlation between genomic mutations and signaling pathways the next lecture will be the continuation of mutation and signaling by Dr. Karsten Krug thank you


Ambassador Senegal? Yes, good to see you, thank you very much. And Madame Ca? Madame Ca, how do you do? And Madame Ca will present his children. They get in the car? Hello, yes. You? Yes, in. Balda, how do you do?  Madame Ca, how do you do? You will come in as you answer. Madame Ca, how do you do? We will exchange our papers here. Madame Ca, how do you do? We look with great pleasure upon President Duf's economic reforms. And we want to continue to be of as much help as we can, particularly with that locust and drought problem that you're having. That's a very serious problem, thank you. Madame? No. We're going to take pictures of the children. Thank you. I think we should be between us. The children, yes. Yes, yes. Yes, maybe you want to be over there. Okay, thank you. Welcome to you all. What's this? Master Bulgaria. No, Mr. Ambassador. Thank you. Thank you very much. It's good to see you. Thank you very much. Many greetings from Madame Ca. Well, hello, Mr. Seed.  And then after we've exchanged papers, you can join us and we'll have a photo of Deputy Secretary of State Whitehead returning to us. He's very encouraged that we're going to have better relations. Thank you. I think that will be just fine. Many good things in the future from our business. Thank you. For this country, for the mankind, for the peace, for the prosperity and so on. Well, he's very kind. Please give my regards to him. Thank you. Please, to have you here. Thank you. The Ambassador of Guatemala. Yes. Mr. Ambassador. Mr. Reagan. Glad to see you. Nice to meet you. Madame Rormosaire. How are you doing? Hello, Mr. President. For me, it's very important on that. Well, thank you. And so absolutely divine. It's Irina. Irina. Irina. She's going to go to see the cameras. It's Maya. Hello there. And? Hello. And Mariana. Mariana. Mariana is.  Oh, yes. Thank you. Thank you. Mr. President. Mr. President. Mr. President. Mr. President. I was especially told me that he is very pleased with you because of the support you did to him as soon as the 11th attempt could be done. Well, we also, we remember with great pleasure, Mr. Rezo has visited our country and know that it hasn't been too long since Guatemala adopted democracy. Yes. We are in the next way. Yes. And we have to improve democracy in Guatemala. We are sure we are going to support it. And we are going to make an homage to you in the confidence of the Caribbean basin at the end of this month. We are happy to support this idea to homage to you because you are the creator of the idea of the basin Caribbean and the ocean. Right. We will exchange our, thank you, and I think your lovely family should come in here. Yes. Thank you. Thank you. Thank you. And I will love it. Oh, thank you. Thank you. And I will love it. Oh, yes. I come on babies. Oops. There. Please. Here. Thank you. Well, welcome. We hope you all will be very happy here. Thank you, Mr. President. Thank you very much. Thank you. Thank you very much. See you all. Thank you. Thank you. That is the first thing I've ever seen. This is the last thing you've ever seen. Just a star. Thanks again, Mr. President. Well, thank you. Welcome. The Ambassador of Tunisia. Mr. Ambassador. Mr. Ambassador. Mr. Ambassador. Please. Please to see you and to have you here. Thank you, sir. Thank you. And Mrs. Hamzal. Mr. President. Mr. President. Nice to see you. Thank you. Can you take on the fireplace and exchange our documents in the new position? Are you pleased with the relationship we have? The progress of your president is turning to a free economy. The president and I knew it would be back in September because of his schedule and reasons. We are proceeding at a fast pace, changing things that were overdue for change. Yes. The direction of liberalization of the economy, democratization of the political process, and election, presidential, early presidential election and legislative election, which will take place in November, have even been advanced to April next day. Well, and we hope that we can be of help with that terrible drought and locust problem. Yes. All right. And you're welcome. Thank you. You're welcome. Thank you. You'll have a very pleasant stay here. Indonesia? Yes. This is your daughter. Mrs. Romley? Yes. Nice to see you. Is this their daughter? No. Is this a student? Yes. And my son? Yes. And another daughter? Nobody here. And son? My second son. Hello. And the other son. Nice to see you.  Well, Nancy and I have a great pleasure, the warmth and hospitality of President Mrs. Swarto two years ago when we were there. And we are very pleased with the relations that we have here between our two countries. Mr. President, we are very pleased with the relations that we have here between our two countries. Right. Pleased to have you here. Thank you. I say I know that our relations are very fine between our two countries. And President Trump, I think his personal best wishes will be fulfilled. Well, and from ours to you. Well, I know that our relations two are going to continue in the same happy way under my successor. Thank you all. Thank you. Like this we had to wear. What's that called? Batik. Batik, yes. Batik. I had the worst. No, George Schultz had the worst one. He won the show. He won. President, the ambassador Sierra Leone. Yes. Mr. Rambez. I'm here to say hello. Good to see you. And his son Benjamin. Hello there. And daughters. Hello. To you. Hello there. You and I will go over in front of the fireplace and then you others will join us for a picture after the meeting. the relationship that's been established for the visit of your President here and I know the changes that are being made in the economy and all, and we look forward to a very fine relationship. Thank you very much. We're looking forward to an improved and closer relationship with the U.S. from Mr. President. All right. Economic change is being made for the better, so we encourage you to make sure you stick glasses sometimes to the glare. All right. Would you like to stay right here with me? Thank you very much. Well, I hope you'll all be very happy here. Good bye. Congratulations. President, the Ambassador of Malawi. Yes. Good to see you. Good to see you. Good to meet you. Thank you. The Ambassador introduces children. That's my son, Bora. That's my son, Santiago.  Yes, sir. Thank you. Please to meet you. Hello, Mr. President. Pleased to meet you, and I'm pleased that we've been able to be of some assistance. Thank you very much. You have been a lot of assistance from the government of the United States. We appreciate this very much. Well, please give my greetings to you. I will. Thank you very much.  All right. Thank you very much. Now, if your daughter should be between us. Yes. And maybe a son on either side. Thank you very much. Thank you. Well, welcome. Thank you very much. Thank you very much. Mrs. Valtissade. How do you do, my fellow? Natalia. Yeah, Natalia. And another daughter, Sonia. Sonia there.  Okay. And then the family will join us for a hope. Nancy and I were most pleased and enjoyed our visit that made us very happy to have both the government and the people were impressed by the visit. Yeah. I noticed they were on the streets. Yes. And it was all too short of a visit, but we did enjoy our stay in beautiful health city. Exactly. The city is not that big, either. What? Two days is good. Well, I'm most pleased about the relationship between the countries. It's an excellent shape in our view, too. Shall we exchange our papers on the right? And now, let's have your family in here. I think we should be between us. Yes. Enjoy yourselves. Find the pleasant here. We're delighted to have you. Thank you. I don't speak yet, but it will come. I can see either of the cameras. That's going to be fascinating. Yeah. Well, it's a great pleasure to have you here. Thank you very much. Very best regards to you.


We are back. Hama unase maje veniako Mr. Kujichocha? We bring barabak after that break.  There is none like this guy in my life. I know you had missed me. I see kwaubaya. I truly have missed you. So we are grateful because we have a panel. A panel of guests. Yani awani wadzi waunguvu sana. Na tunanda na awa on na jita the brand. So kwa ni the brand. A lafukuna wengina na the brand. So today we want to really know what is this about the brand? And what is it that they feel that they are brands to influence the brands. So without further ado, tukaribisha wa geniwe tuwaleo kama bawa they are known for better or amani lack of words, social media influencers. Tukonama weji wei nyewa. Tafadali wadzi tambulisha kariboni sana. Sana, sana, sana. Together na ehm the brand. Ehm the brand. I did introduce eh. Kama zina ni lean the brand. I am the online strategist. So. A mina julikana kama spikes official aka the voice of influence. Ya ok. Mina julikana kama jena the brand. Just jena the brand.    So utatanya wuwe tu ambaia you guys all know she is the brand behind jacy circle show and of course at one b ni ni ni na fa niya, as far as influencing jacy circle show is kwa sana. Na bide, buna berojtuko kwa followers matu, badafika maka. Mika masasa, leo ndiwile si. Ape rei, soli ake. So, leo me wa moku niya kakwa ni. Nwa wato mwiziki yangatiki. Tukiku salimi a tukima. Ako na shida. Social media influencing ni wide. Actually, aiku wangi about the number of followers peke ake. Number of followers ni kama 2 a by the way. But it is not the basic thing. So, eki kuja kwa J.C. Circle. Mi hu focus more on the interaction. Ya, the show itself na mafans. Mafans kufili wa warmth na hosts. Kufili ni kama, wa me impress. So, wa fil ni kama show niyao. They are part of the show. So, it is more for J.C. Circle, it is more of interaction. Rather than even the followers. Although followers is a plus. So, but kuna followers unendangakiki gain pole pole. Kuna piyendi a ku gain a raka of wheat, a ita makes sense. Because hawa ta kwa legit. I know. Nene nene nene nene. Nene nene nene. Palinumafika. Tukitaka tuku fiki shia 100k. Tuna yaza kuku fiki shia. Laki ni a ita kwa na. Laki ni a ita kwa na maana na get. So, kuna ili a ku enda raka unahit 50k at once. But haida nene ma impact kuba. Nakuna ili tuia pole pole, but you get legit followers. Siwa lambo wa naku unfollow. So, do you after a while? Eh, kuna hawa. And then, sa atuna a itanga kuna ghost followers. So, unahika post, hawa like. As in, ukotuna namba. You get, hauna interaction. Ukotuna namba. I see what you mean. Tumikabi sa. Ayasawa. So, sa tundu kwa the voice of influence. When you are certain owner. Thank you so so much for making time to be here. We don't take it for granted. So, as far as you are concerned. Women say, ukatu chocha, ukatu chocha. Itka kukuna spikes to even turn and post notificate. And you say, wukuna mawa. Come on. Game, game, game, game. Tuka jengwa. But then you have to give a game. Tuka jengwa. Follow, follow, follow, follow. Anafu kicha, ika enda kwa. Ika toka. I think kuna, Unejiwa kuna bile, Brands ataka hintu kwa ambiwa hibila. Kini, kuna teima nikuwa na. And you don't have to spikes. So, ni spikes jzei kwa enda. Ika toka hintu kwa enda. Ika, nika. Ika toka hintu kwa enda. Ika na ika na, akuna followers na game followers. Easy to be haja kukuna. There you go. Sasa toa ambiwa. How? Because we are still following you. And you are the voice of influence. And you are definitely influencing us in so many ways. Lekini sasa tuna gain aji ju tuna fil mi brandi ya kundun. Nende liya kupush. Nende liya kubrog. We follow you kunao. So, sasa ti siya. So, gain through you. Ti siya na wataza maji? Nene nene nene. Thank you so much for this opportunity. First, ngependo kusimaiwi. Ule liya na meseima. It's not always about numbers. The reason why neli yacha, it's because we realize, okay, yes, people think being influential is about getting the numbers. Of which they are wrong. And meseima kuna ghost followers. Kuna wala tako follow lewa wakwan follow. So, my perspective of influencing is to change someone, someone's life. The reason why I stopped, and if you are keen on my post, I create memes. That makes you happy. I create happiness in people.  Because, mimi sijiwia ble niko, sijiwia ble uko. So, through that happiness perspective, at least there is something that will change in your life. There are other things. There are so many things that affect our community. So, me uchukwa that platform, since I have the numbers, yes. And I have also the influence. So, I'll use my influence to change the community, someone's perspective of life. Yes. So, that's why now through Spikes Visual, yes, now Brandyako, as a camera person. Yes. So, these are two clips, not two kiyanga, or what to now watch, okay. Ile uki uki watch. Of course, there is something ita rangkwa mindyako. Na ita change. Very true. Okay. Okay. Wow. That's an insight. Guys, we're doing this very, very fast. Sorry because we're pressed for time. But I wanted to take every, you know, nugget you can be able to get from these guys because it's not doing justice kwa leta palafu huja jengeka because unlike, you know, unlike me, penginewo kuna jua, but min kuna jua. Hey, I need numbers for me to be termed as, you know, an influencer. So, to me learn nivutu zingine si lasima two numbers, it's all about interaction na wene una rich kwa watu. Sasao? Aya. Last person. Jenna, the brand. Yes. Sasao, wewem dem. Uka nana na ume build brandyako na kila kitu. Congratulations for that. Thank you. Thank you very much. Sasao, as Christians, tu na kwangana challenge kubu asana because as an influencer and you know social media do kumeklata kila kitu apu. As Christians, ni nanbeliva, soda tuku apu na tu na kwangalia, no na tu influence. So, how do you align your influencing on social media irilet na principles na God na wezeku influence watu in the right avenue Goda mekupatiyo platform. Okay, first of all, thank you so much for having me. And then about the part of turning my followers into something influential about the Christianity and all that. First of all, when I was starting, I didn't know where to start and how to reach people. So, I aligned myself with the people that are involved in there. Go spell industry. For example, size 8 and all that. So, through them, I was able to influence other people and be able to tell them that this is the right way to follow and this is which way to follow. And also another key point is that you have to take someone that you can look up to. For example, Sally right now, I can look up to you. If you know that Sally is the person that inspires you, follow that person, tap your blessings from them. You know, there is sometimes God cannot bless you directly, but he can bless you through people. So, if you can be able to tap your blessings through Sally, that is everything. So, if also Sally you can use your number to influence other people, telling them the word, also telling them that it's just not, your blessings cannot come tomorrow, but if you hang in there, that's it for you. And especially to chipping to that? Ya. Actually, my totally agree now what she has said. Yes. Relating to social media and Christ, we are all there, kinamutako social media. So, what is happening? A lot of things are happening outside here and there is no one to talk to. Always, if part of them say mwenyako willing kwangere, yes, we have the main streams. Kunawe langine, they might not talk about it. Not everyone is able to access the media. So, social media is a digital platform whereby kinamutako. So, why don't you use that platform to spread God's word? Ya. So, Sally, kunae lakwangere apu? Not really. I think I'm a killer kid too. Okay. So, now we come back to you mama. So, here you are and that, we can all changia. So, JC Sakul is a brand. Sally Nyuike is a brand. DJ Nix, the spin doctor is a brand. So, not forgetting, lean the brand, badu, you know. So, ni tactics gani and by there you don't need to share kusama ini secretiangu, but feel free. What are some of the tactics that you guys use as influences to push a brand? Okay. Ambo isikue ni wanyotuna sema, ghost utlone, kanatu ni numbers, but actually brand inafili ikuna influence kiasi, kina zanzanaya? For me, the major, I can say the major tactic that I usually use. You know, people outside there, kama sasana, for example, sasa Sally Nyuike ni brands. So, kuna mtu pali inje ana kanga, ana watching JC Sakul ana feel, Sally Nyuike ni brand kubuasa nasiwezin fikiya. So, now, the basic kama the major thing ni ufa anya ni kumekshua, like kumekshua, ule mtu pali inje ana feel.  Social media. Yes. And you ina working as a plus for your brand because now people will feel like nga jwa pali inje kuna ile noshonia so and so ni start anongi angi na watu wata tukimongelisha na tuwa sio, minu. So, my work is to make sure your work is easier. You get. Nekushika. It's to make sure your work is easier na kumekshua kuna ile warmth between Sally na watuake, between nicks na watuake. And then for JC Sakul, kuna ile warmth between the show na watuake. Same na linda brand. I think what one is as far as other what's up people na muna fanya I know. I know. Responding to all the messages when to a blue tix. No, no, no. I am not a blue tix. Blue ticking, blue ticking. So, you end up as a social media. Muna konfang. Oni ksa isoma. Okay, but kata jivu. I just say my name. Me, me, nim tuwa pali. What are spikes? Text messages. Spikes said to jivu. Okay. Me neza se maivi. How I work with brands. See your brands out in Tafania Kazinas. Okay. I have my own personalities. Same to you. Like Sally as a brand, Nik as a brand. Lazi ma ni understand your personality. Is your personality matching with mine? Then you can easily work together because brands working together is like marriage. Yes. Agree upon something, then we come up with that good solution at the end. Yes. That's how I work. Very simple. Awesome. For me about building brands and working with other brands is about perspective because if you say about influencing we are all influencers. If you look at it at a followers perspective because if you have 50 followers you can influence 10 people to go with you to Nakuru right now. That is influencing. But the question is what are you influencing? So that the way spikes has said the people that you choose to align yourself with. If you want to align yourself with people who are like parting every day that's fine. We want to judge but for me when it comes to brand I have to understand what am I learning from you as you learn from me. It's not just jumping into the bandwagon so that people can say you are an influencer. No. It's about understanding the perspective and what you bring to the table because when you understand what you bring to the table you are never afraid to eat alone. Wow. Deep. Very profound. So really quick and this you know one of you can decide never turn the money so it's like is it a calling or was it a decision that you guys thought you know what I'm actually an influencer on social media let me just do this and then wukaansaku build so either I can say the guy can start here. So for me na the same it's a calling actually the name voice of influence nili toka church Yes. Nili feel. Then I was told that you need to do something about the acts and entertainment industry in Kenya. A lot is happening and no one is speaking about it. Yes. You have the numbers you have the influence use your influence to change the music industry. Yes. Okay. Ladies. For me for me I wouldn't say it was not a calling but it was something that I was so passionate about. It started come up a passion. So I was really passionate about it and all that then I did my research and everything na nika juakumbe is a way that you can influence people outside there positively using social media and then another thing came as a plus of which I ended up working with people or gospel industry a lot. So it was something nili kuja trivokama passion of which now right now as a way of influencing nimi unangalie witu zina happen kwa industry and all that but una feel like akuna people are not talking about them ama unadari buku find solutions to them na nini nini wuna kosa so in dioma na kuna something na fa niyanze at least nina tackle I'm starting something nina tackle lizo witu witu zina affect zina affect I mean sorry gospel industry as a whole because what you pala injew anangalia gospel industry in dioma prela you get so na umtu akimsali akimes for instance na kwa kakwa shuia gospel buna na fa niyanze nina eza fa niyanze so I can start I want to start influencing now guys through social media but via something totally different in account witu yes yes zina okay for me I think influencing and all that social media is calling because if it was something that everyone would be doing it and yes many people have likes and you have followers but what it is that you do that it is unique from everyone because for example when I started I didn't even have a smartphone I used akamuli kamuizi and there is something that I believe that one day God will show me the way and it's through understanding your passion and knowing what you are doing that's why we are here today because for example Instagram Instagram was so lit in 2016 yes these people can agree and Facebook was lit on 2014 2015 and 2016 but Twitter is it has never gone out of date so what for example the people that knew in 2016 that you can gain you can follow and you can unfollow they have followers but right now we have like 50 likes and 100k followers but if they knew that it is about passion maybe many of them wouldn't have fallen out like right now but why are we still here because it is understanding what you are doing and why you are doing it and it's appalling so after all sitting down we are humans in there we definitely gotta gain something and I'm talking about finances so how are you benefiting as voice of influence the brand pushing your brand on social media that is a very tricky question because most people don't understand that being a social media influencer is something that apart from it being passion it has to put food on the table at the end of the day so many people will be like fanyakazi mungu but at the end of it if you people are being paid to do what you are doing in the case that you are being paid why not because at the end of the day we all have bills to pay and there is something that I always say you cannot expect your friend to be doing everything for you for free all the time they have bills to pay and they cannot do it if you keep asking everything for free because you are friends so understand which brands can build you as you build them as you are gaming I will say I will take myself as an owner of a shop you two guys are brands Nix unataka maziwa kudukayangu Salu unataka mukate kudukayangu utaku yatuni kupe it's that exchange I benefit to benefit at the end of the day we all rise up towards a certain standard awesome I will echo Genad's words because that has been my biggest challenge I know yes to thank Christ to Gengani to Gengani Sisi wanted to fanya to fanya kaziama now people should understand mainly fight naios because what you have to understand at any job like what you are doing is you are asking if you are going to buy a shop if you are going to buy a shop I will raise at the site I will tell you how much you posted on Instagram and I will post so the question is what will you post you get what will you post what will you post what will you post this is a job so I am supposed to get something from it as we appear to get something from it now we are going to fail sana by the way hit it sister hit it hit it want to fail sana want to cut to a support but how a turkey cut to thought took on a needs took on a needs you know they want to mihaile God is blessing talk of corporate sana the same question talk of people now who are not brands but not individuals but company corporate how do you choose who to work with considering Pia kwa aligned kwa muna choosing yaji how to work with those people okay when it comes to corporates I'll say again let's manage as a corporate what are you offering what is it that you're selling out on the market as you are selling it out is it for the general public because I understand each and everyone of us being different influencers we have different categories and different audiences so as I'm taking your brand is your product or service equal to my type of audience that I have yes I think about that I'll say don't be so thirsty for opportunities that you drink from every cup that is handed to you because their companies for example right now I know a company that has been looking for a social media influencer in turn since 2015 they have been recruiting interns ever since you know that is the way of looking for influencers and not paying them so you should also be aware of the people that you are choosing to work with just because they are handing you a plate doesn't mean that you need that meal true awesome me same too as in you should always say work because vile tu jena mesema me bia tu kuna when yena jua actually actually there's a company that I know kazi aw ni kupik influencers wana kuchkwa tu amekwandika then you'll work for them for a month they'll not pay you and then of course tu spolipo wana achajob after the next month uta ona wami picking influencers mungini and that also goes to us bia as influencers ni apu kuna kujia piyana yo yo vile jena mesema yo you should always be careful wuki choose ni ni and then know your work na jua wuki piki wana kampani moja u drop iwe tena kampani ngini iku pick iku drop at the end of the day kuna shida sawa sawa being influencers it's not an easy task I must say and like you all said if it was something easy at a system kutuna faena lakini we have only established one thing it's a calling and because of that many of the times that you have to tackle so many things and unfortunately a lot of it one in negativity because apu wana kujui you know and so you receive the wrath of so many people some will you know condemn you some will judge you in so many ways but then the beauty about it is because we know Christ and Christ is in us we are able to find refuge and us as JC Sackleshaw for us is to stand with you and as we trust that the Lord shall use you more and more and yes that may the Lord continue to shield you and to protect you and I'll speak Sam's 91 and everyone of you especially the verse that says verse 14 because he loves me the Lord knows that you guys it's a calling and he knows that you love him he says I will rescue him I will protect him for they acknowledge my name they will call upon me and I will answer them I will be with them in trouble and I will deliver them and honor them for with long life will I satisfy them and show them my salvation may you experience that every day of your lives it's not easy and as you lift up his banner may he also be there for you and you be able to experience his salvation in Jesus mighty name thank you so much for making your time and being here definitely given another opportunity we shall be able to chambua zaidi this one clearly indicates and this is to you mama our producer we definitely need from beginning to the end you will see who immediately we will see we will see because there's a lot to learn guys from these people and I'm sure we'll learn nage to one or two just like I have so now I need to be able to conclude the show but kuna kitu ni kukatsho they have not shared their social media really at a way unata kunata media jao ya sababu what you want at our follow pal you want to follow we want to be your audience me I want to be your audience then I will see as you can clean the brand clean the brand at clean the brand kila mahali clean ni L Y W N E wow at specs official platform at Jenna the brand everywhere nice alright they have really knew what to do so neither konata kunata follow right now but anyway we need to come to the end of the show and this one is really hard because personally for me when I say end it actually means the end


Good afternoon Wilfrid Spiegelberg and Peter Boschko here We're part of the PMC unique Apache unicorn group Designed the Apache unicorn from scratch I've been doing Apache unicorn since inception about three years ago and Peter Hi, my name is Peter Boschko Join club there in 2016 worked on Uzi had to produce yarn and then join the unicorn team in 2021 So what we're going to talk about today is how we are using the schedule framework to give you some batch extensions In the Kubernetes environment. So like I said, we're coming out of the hip Hadoop yarn area so batch processing large data processing and we're trying to give you the similar kinds of things within The setup around Kubernetes so What are the things that we're looking at when when we look at? Scheduling on on Kubernetes We want to do workload queuing when when batches get deployed when batches get generated We generate often or more than one go We want to keep them around we want to start them whenever things will or need to be run We want to do an all-in-one kind of scheduling setup gang scheduling. We don't Spawn up one pot. We want to do a set of pots, which one is the driver others are the workers Spark is a good example for that, but there's other pie torch I'm Peter's there's a number of things that will do that for you things that are not available in instant thing the other Extra bit that we want to give you is Application sorting so instead of looking at just one pot or a job or a demon set We say Nana. There's a mixture of pots a group of pots that together form the application We want to schedule based on the requests that come from from that thing it could be One pot to begin with Scale up to a thousand pots and then drop down again to a couple of pots Like what we do with data processing at the point in time that we need it We we scale up the pots. We do what we need to do and the pots go away. So We've got a real bursty kind of a deployment But we we still want to see all these pots being scheduled as part of one thing not every single pot separately so these kinds of schedulers these kinds of Facilities are there when you look at high performance pro the HPC pros and the batch processing when you come from a slur Or a yarn kind of a setup help These kind of same kind of things we're gonna do or you want to do from a Patchy unicorn within Kubernetes, but Kubernetes from its origins was always a services based Set up We want to do both at the same time There's a number of schedulers that will give you some of the batch things that you want to do But then they don't do the services or the other way around when you use the default scheduler You get the services, but you don't get the batch Things that you want to do so a pitch unicorn. We schedule whatever you give us so we put a application on top of the existing pot objects the Applications the demon sets whatever you want to run on on a Kubernetes layer We we run on that and then we also allow you to easily run spark jobs tensorflow jobs MPI jobs Whatever you want to do without needing to change the underlying framework to submit your jobs, so we don't want to go in and needed to run a specific tensorflow setup or Compiled from scratch because you want to do batch processing or the same with spark so we We're gonna give you a simple integration based on the minimum amount of code changes Preferably no code changes on the way that you submit an application the way you submit a job and purely use annotations and labels on on pots or Demon sets or whatever you submit to schedule and give you the batch scheduling things On top of that when you start looking at data processing there's always The question where do I run it? How do I share my? Resources that I've got in my cluster When you look at the default scheduler you've got one scheduling queue So a pot submitted by the first user or by the 10th user It doesn't matter they're all coming into one queue and it's getting scheduled based on a priority kind of setup most data processing that you want to do is not looking at One user of five users, but you're looking at hundreds of users. You want to share the quotas you want to share the system nicely so Unicorn provide a Hierarchical queue structure to place these applications in and do you make your scheduling decisions? So instead of having one single queue that will contain all the pots for the whole cluster You now can subdivide and schedule subsets of the pots based on their own rules and their own Priorities first in fact, whatever you want to do Within that hierarchical queue system we give you the possibility we give you handles to say These These queues can only run an X number of pots or an X number of resources But we also want you to be able to say With guaranteed this queue gets Half of the cluster or it needs ten of the nodes always So every single point in that hierarchical queue we can give you guaranteed these sources Quotas or different scheduling policies. So going from one Possible policy throughout the whole cluster. We now give you a Set that you can self-define and self configure over the whole cluster When we started off Working on Apache Unicorn There was no plug-in architecture in Kubernetes. So before we when we started the There was the only way that you could really Change things on the scheduler was completely replacing the scheduler. So we wrote our own scheduler we implemented all the functionality for Binding pots doing all that kind of stuff because that was the only way that we could do things The next step was the extensions that were there for Doing HTTP call outs and then you could customize some of the behavior That didn't perform well. So that was pulled out of the Kubernetes code and the next step that they came up with was the plug-in architecture Within the plug-in architecture, they'll they gave you a number of extra things that you could play around with so Unicorn went from the standard deployment like what we're running now Which is a complete custom scheduler to a plug-in architecture, so we started off with simple Core that does all the unicorn core that does all the scheduling decision makes all the scheduling decisions And then we've got a shim that integrates with The API server pulls the information in does that's all the things with the pots and what we need to do And we've got an admission controller sitting on top of that to do some of the more advanced stuff making sure that I think It is as easy as possible for the end user Moving to the plug-in version of that we changed over from Doing everything within the core to having the shim that now completely includes the default scheduler so instead of Writing all the code to bind pots to the node and and do the volume binding everything ourselves We now rely on the default scheduler to do all these things for us So we hook into the scheduling framework We implement certain points in there and we augment or you replace whatever we don't want or what we Want to change from the default schedule? So This is the picture that's probably have For people that have been looking at the scheduling and the scheduler in Kubernetes that has shown up before The right-hand side the binding cycle is the last point in the whole whole cycle That's where we leave things The scheduling cycle that's already been there that's been there for a long time That's What does the real? Checking which node we want how do we place it on there do all that kind of stuff and then Part of what is just come out in 126 127 is the pre and queue Plug-in side of things so we worked with the Scheduling sig and we said look we want we want to be able to do a little bit more instead of just having all the pots Flow into the scheduling cycle immediately. We want to be able to gate the pots in a pre and queue cycle so That has been delivered as part of 126. So how does Unicorn use these Plug-ins and where where we sit we've kept the unicorn core. So our Core scheduling code has not changed. So whatever we deploy Default mode the standard mode or the plug-in framework the unicorn core still makes all the scheduling decisions The only way that has changed between what we do with the plug-in and the default mode is the way that we interact with Kubernetes and What we need to do ourselves and what the default scheduler does for us so the first part that we implement is the the pre and queue Plug-in because we want to be able to decide which ports the scheduler looks at and which ports that get put onto nodes and and be processed so Without the P&Q There's a lot of overhead that flows through the whole system because We can't stop the default scheduler from looking at a pot That's just not built into the scheduling framework and that meant that without the pre and queue hook Pots that unicorn thought could not be scheduled yet the default scheduler had already looked at and marked as unscheduled rule and then the auto-scaler kicks in and says I need a new note because this is marked as unschedulable. So that's where the the pre and queue hook came from Then the second point that we implement is the pre filter the pre filter Runs over all the nodes and decides which nodes to use and which nodes not to use so That in combination with the filter hook Allows us to select the nodes and make sure that the node is selected is used that we from the unicorn core have decided on the last Bit that we implement is the post bind That is the last point in the cycle and when the post bind comes back. We know that the port is Scheduled completely accepted by the cupelet and is getting started by the cupelet So the post bind is a housekeeping point for us because we know that everything has gone through and that we are at the final stage The input from the core scheduling cycle goes into the first three plugins So we decide what goes through decide which node we put on and for that we interact with the three Plugins the the pre and queue the pre filter and the filter With all of this we want to do quotas and all these other things now Peter will go into further all the quota tracking Yeah, so let's talk about quotas. So as we've read mentioned, we have a higher article A hierarchical model. It's a hierarchy of so-called Resource queues That's what we use to calculate the available available resources To the running applications these queues can be created automatically But also in in config in configuration files, which is right now. I am We have to submit applications or pods to the leaf queues. You cannot you cannot run stuff in the parent queue And then whenever there's an allocation or there's a resource request. We do the counting in the leaf queue Increase counters and that propagates up all the way to the root queue So this essentially means that at any point In the hierarchy, we always know always know the the resource usage We know the resource usage at at for every for every sub three You can put quotas on leaf queues and and and parent queues and Since we know the usage all the time It's very easy to enforce it and we enforce it in every scheduling cycle So here is a very very simple example So that there's a cluster for developers and testers Leave queues our name after the users There are some running pods Alice is running two pods and Mallory is running one pod This is the current resource usage that can be observed. So obviously root sees the the total Which is 15 gigs of memory five CP CPU Dev 20 gigs of memory to CPU and QA 30 gigs and three CPU and now we want to put some resource limit or quota on the on the on the dev queue We want to limit resource usage for the developers 60 gigabytes of memory and for CPU and also it's worth mentioning that for the root queue. This is a calculated Automatically when the unicorn starts the nodes are Registered and we just retrieve the capacity and later on we update it when a node join node joins the cluster or Or leaves the cluster Okay, and then Bob Bob wants to run a pod But the functionality for him this pod cannot run at least not at the moment Dispending why? Because of the of the limit. So this pod is asking for four CPUs. There's nothing wrong with the memory but the current usage at the dev queue is to CPU and there's the there's a limit for four CPU and There's the and that's not enough. So so Bob has to wait until the pods that were started by Alice terminate and It's important to know that the Bob's pod is pending. It's not rejected and You will see why this is important Okay, so namespace quotas versus unicorn quotas, but we can also use unicorn to manage namespace quotas Why why would we do that? So if you use namespace quotas We set it The quota is exceeded the pod is ejected end of story and If you want to run your workload and probably you want to run it some time later You need you need some retry logic the script or you try manually And also other users might be competing for for resources to and there is no no ordering between the users So you might lose this race against others so it this is Not an ideal scenario In unicorn you can have a setup Where the queues are auto created based on existing namespaces. So for here here in this example of sales finance dev and test these are existing namespaces and Unicorn created them when a pod was submitted and In this case the pods are not rejected if there is Is there's no more room? the quote has exceeded them the users just had to wait and When there is enough resource then they will be picked automatically based on based on the ordering that is set You have to set so-called placement rules in order to have these queues created And also you have to update the namespace object themselves. There are two annotations that you can use the the quota itself and the parent queue This this arrow says that the development and production queues are optional grouping. So that's not as that's not necessary all The the leaves can appear under root directly. So that's that's not an issue But sometimes this is this is desirable So this is so this is how it looks like when you want to configure this This is by the way from the upstream documentation when it's Explained real well. So on the left side, there's the Unicorn config the yam With in green you can see the static use the the production and the development. This is what This is what I called optional We call them like sometimes configured queues or managed queues These queues always exist. They didn't they don't disappear until you remove them in in the yellow There's the placement rule section and if you have used Apache Hadoop before Hadoop yarn With fast scheduler or also capacity schedule. It's the very same idea. It's also called placement rules there You tell Unicorn how to name the queues that don't exist And and that's it then there are all kinds of rules like name after the user name after namespace Some maybe some labeling. So there are different kinds of rules and These are the the orange This is these are the two intentions that you have to put On the on the namespace object the first is is the quota is this is a tiny jayson and second one is is the parent queue that That tells where the namespace queue should be created and finally we also want to put quotas on users and groups and This this feature consists of three parts The first we have to determine who submitted submitted the actual workload And it turns out that this info is only available inside the mission controller It's not present on the pod or deployment or any kind of other objects At least that's the only way what we found You need a you need a mutate webhook We have a mutate webhook we extract this info and we modified the We modified the pot spec Of the workload not just for pots also for Deployment job cron job accept replication controller. So we had an extra notation This is also a tiny tiny jayson and later we deny changes to this annotation You can you can fine-tune this behavior change it, but that's that's the basic idea And then there's the tracking itself It's it's the accounting increasing decreasing counters when there's an allocation or report termination for the different groups and users and We have a nice rest API where this is visible. So we have separate endpoints for users and groups and if you open the user Endpoint you can see this user total and Then you can see the per queue statistics where that the user is running applications and the usage There's there will be a demo about that and There is the third one which is not ready The actual this is the actual enforcement so this is this is in progress and We targeted this for one three all the latest to reconversion is one two all There's the general ink you can check it out It would be great Yeah, so that's it I skipped over pre-emptions so one of the main things that we were missing in our current setups is How would you do preemption in in this kind of a setup? If you do want to run one single port you can preempt but in this case We've got a full queue hierarchy. So how do we decide what we do with preemption like the default scheduling cycles We do preemption based on priority classes The only thing is that we've added an opt-out from a unicorn perspective So instead of saying that something is allowed to preempt during the scheduling cycle We now say don't preempt me when I'm already running If you look at jobs that are running for instance, if you've got a spark job running or some other job running for a couple of days and You kill the pot that belongs to that that could be really costly So you want to be able to opt out that we allow that We've got the queue configuration that we therefore needed We max is the quota and we guarantee a certain amount of resources, which is the basis for preemption We've got some other nice Really complex things that we can do with fencing so that you can say certain parts of the hierarchies can't preempt other workloads Freeing you don't want to have production Pempted because somebody in start something up in a death or in a test environment I'll skip over all that there's a presentation that was done during the HPC and batch day or from Tuesday that goes into that really elaborately and then shows you with a full demo demo on the bottom of the slide is the link to that presentation so Preemption within the system we do that again as part of the normal scheduling cycle and we use Guarantee resources for that so again within the the setup of the queue system We allow you to specify certain points in the in hierarchy and we Give you a guaranteed amount of resources in the setup So like a multi multi-tenancy fencing with priority offsets or Nice fancy things. There's a lot of documentation on the side Goes bit too far into them What we can show you and we're gonna try a live demo here based on a kind cluster will show you how we redistribute some of the Some of the workload between the different Queues and What we what we do? I've set up a small kind cluster to do that for a three Node cluster and I'm going to submit a simple application the application is Just a normal Job with a couple of extra annotations on there from Users Too long to show what we want to do and where we want to run it Let me create the application So we create the application we start up ten pots because of the quotas that we've set up We can't run all of them We can run eight out of the the ten the other two pots will just stay there and stay as pending in the unicorn web UI and now What's that one we can we can see that kind of information to the web UI is gone Okay, that's what you could we could do a live demo that's ma'am We've set up the the queues we've got this all running it runs in a low priority setup and Now what we're going to do we're going to submit a high priority job Against it on the same system. It runs in a different queue and It creates a number of high priority pots These pots at first day in pending because we don't we're not going to directly preempt things We wait a little bit to see what what's going on and then after the wait time is done We are going to look for enough resources that will allow us to do what we want to do so 30 seconds is is the the setup and No Come back and in the meantime, we'll see that we have redistributed some of the data. So here's the unicorn web UI We have created the the queues We've got a root with low priority root with high priority and Within the queues we have got the setup that we had first and we have redistributed some of the load from root load to root high And when we reload we should get the final state And over time we have moved. We've killed some of the root low. We moved that into root high We've got a guaranteed quota Sitting here That means that we are going to preempt whatever we want and we need but we are never going to preempt more than That we allow to so we all we've guaranteed that root low will always have that's this These spots running so we will give that we don't go below that and that also means that root high which had a higher guaranteed but we've only got eight CPUs can't get to Allocated to the the maximum of the guaranteed that is there so We still have got pending pots But we've redistributed Some of the high-priority are running some of the high-priority are still pending. We've got low low priority Pending because they're still not being scheduled. That's just sitting there waiting for for things to happen To hook that back in again to what Peter mentioned around The quotas so we've got the quota tracking going through The load for Peter runs in a root low. We see exactly what's happening here The second user is also tracked root high We get the memory we get everything and then from the group. We were assigned. We both had assigned the same group We see that both applications attract under that Same group with root low and root high so the total usage is there For everything everybody to check Now if I now Delete the first application We will see that the left over pots that were pending for The scheduler will get scheduled and will become running after a short amount of time. So We pick up where we were and we we schedule whatever we have got as a leftover amount no That shows what we do with this within the scheduling framework We've got full control over what we do with pots what we assign which gets scheduled where we place them because in the kind cluster that's a little bit more difficult to see but We have got full control over what's what gets played where and We also are able to do preemption based on The quotas what that you set up that you want so from a batch Processing perspective that gives you a full control over your environment You can say I want certain users to have priority over other users. You can even say Peter is not allowed anything in in one queue or the other queue so you can do a full blown Multitenant environment within the system that we're going to set up So this was the cluster that we used we had a prepared cluster quotas and Pemption The applications that got submitted were one for the use Peter one for the other user and we saw the redistribution of the quota to the other user and We hopefully have some time left over for Q&A if there's any further questions Yeah, there's a microphone aside for this for the people that are streaming because I was told that there were a couple of people that was Streaming so if you could Go to the microphone, please How do you? Say like with driver and executors in spark. How do you stop the driver being prepped it? So The the main thing is the priority class that you set on on the driver And within the priority class you opt out from the Preemption so that's that's the first thing the other thing that we do is when you submit Spark driver and other things We create an application object within the application object We mark the first pot as the originator pot. That's the one that gets created first the also the originator pots Even if it doesn't have the allow preemption tag set always gets put put in the back of the queue for Preemption, so we try our best not to preempt the driver pot It's not a guarantee.  Yeah Thank you and The demo you showed yeah, well those two applications running with the same priority class. No, no, so they were running with a different priority class One was running with Low priority that was the one from in the in the first queue that we set up and the second one was with high priority If I would have submitted a third application While the first low priority was running and I would Put in a third application also with low priority The second application would have been scheduled first Not only because it's a higher priority, but we also put five four on the system So first in first out, but you can decide that Anywhere in the queue hierarchy So if you say I want to schedule Certain part of the queue First in first out, but the other part of the queue. I want to schedule Fairly or purely based on priority. You can do that That's that's all set up and all available within the queue hierarchy. So it's not just one policy It's policy per Part of the queue that you set up. Okay, so one more question So if I had two applications to the same priority class running Well, I ran one and it had a guarantee resource of x, but it filled the cluster And I run my second one and I had this guarantee resource But the same app the priority class would resources preempted from application one and given to application two Such they both had at least guaranteed. Yeah. All right. Thank you Yeah, if they're exactly the same priority. Yes. Yeah, we will still do that. We distribute again Over over the two applications I have two questions One is how does it is compared with queue the queue with k that was also being shown like If you can broadly see the advantages and disadvantages of of both and the second one has actually to do with this Part which it was not clear to me if you wanted if you have some kind of implementation for fair usage And if it has like something fancy like the the classical job schedules do which can kind of even how much have you used in the past with the kind of A lifetime kind of We we don't look at lifetime so to say so We because we don't know When you run for instance a spark job or whatever we've got no idea what what comes in You can ask for gang scheduling But you ask for a certain amount of resources We try to do our best and give you all the resources that you ask for if we can't You as the application submitter can say Do I want to proceed with whatever I Can get or do I just want to fail so soft or hard gang scheduling So that's the only thing that you've got when when you look at that side of things Um Compared to q I think q is not Hierarchical so it has a set of cues but The hierarchy and and the the way that we distribute with guaranteed The other thing q when it does preemption it preempts the whole job It can't do one pot at a time. It it can't do distributions like that it it Does the whole job or nothing? So that's not as flexible Sharing quotas in the When you look at what we do in A set of like you see here We can share quota from qa All the way to the other side That is not possible with q You've got a group of group of cues that you've set up before and so The flexibility is not there as much within the the q Set up Thanks a lot Any further questions So when you leave the pot impending Is the application able to determine that the pot is impending because there's not enough resource available And it's it's effectively cued Compared to it's impending because some secret it needs isn't there or some other reason why it might not be impending Yeah, so we we use the event system and we put events on the pots to show you that the Pot is waiting for resource to become available. Yeah, we do that Yeah for sure. Thank you. That was it then


Fel cyfweld dishesnaethau arferng ha tri oconf prematurell. Here you are. Arthaf, mae y bydd y cynhyrch chi dechrau am hollu iawn nag sylff, maithFinally for those with conditions such as MS. Adam disability payment was developed in close co-operation with disabled people. We have made several improvements to provide a far more positive experience compared to personal independence payment, which it replaced. Social Security Scotland seeks to apply the eligibility criteria fairly and consistently to get more decisions right first time. We have been raising awareness of the consultation on the eligibility criteria for the mobility element of adult disability payment through stakeholders, including disabled people's organisations and consultation events. We are encouraging people to reply to the consultation and tell us about their experiences of adult disability payment. Fulma Griga. I thank the minister for that answer. Does he agree with the MSSI to have been in touch with me regarding this issue that assessing an individual's mobility cannot solely be evaluated in the ability to walk on a flat surface for 20 metres as that does not take into account variable surfaces and fatigue that takes place after the exertion? Will the analysis of the consultation take that into account? It is essential that Social Security Scotland makes person-centred decisions that take individual circumstances into account. That is why Social Security Scotland case managers have training and guidance to consider the individual mobility needs of disabled people. That includes how to take into account the impact of pain, fatigue and fluctuating conditions, starting from a position of trust. The consultation asks people to give their views on the eligibility criteria for the mobility component, including the 20 metre rule and the impact of fluctuating conditions. We encourage people to contribute to the consultation and the analysis will reflect people's views on this matter. A couple of supplementaries first. Jeremy Balfour joins us remotely. I ask the minister if the rule was extended when mobility rule was extended from 20 to 50 metres, how much would that cost? If he does not know the figure, can he write to me at a later stage with that figure? I am happy to take the point away on behalf of the member. Those are important considerations with regard to the financial position. However, at the moment, I encourage people to reply to the consultation on the issue. Pam Duncan-Glancy. The Government promised that there would be pre-consultation engagement with stakeholders, but conversations that I have had with those in the sector suggest that this did not really happen. Can I ask the minister to set out more detail on any pre-consultation engagement and which organisations the Government engaged with? We engage with stakeholders with regard to adult disability payment on a regular basis. The most important thing now for all of us for the collective common good is that we encourage people to contribute to the consultation. I know that Pam Duncan-Glancy has done through our social media. I thank her for that, and I encourage others to do the same. To ask the Scottish Government what progress has been made on delivering the beauty house agreement commitment to ensure that community housing trusts are adequately funded so that they can support the delivery of enhanced rural home building plans. I am delighted to say that we have made significant progress in delivering this commitment, and last week I met the community housing trust to provide an update. I recognise the vital support that the organisations are continuing to provide in rural and island areas to ensure the delivery of the right homes in the right places that will meet the needs of communities. My officials will continue to progress this as a priority strand of the remote rural islands housing action plan, and I will provide further details shortly. I thank the cabinet secretary for that encouraging answer. Does the cabinet secretary agree with me that community housing trusts play a vital role in making sure that affordable housing and placemaking is delivered in some of our most rural remote areas where the addition or return to use of only a small number of homes can make the difference to whether a school or a shop stays open or closes? That is why the agreement between the green MSPs and the Scottish Government attaches such importance to securing their funding. I fully recognise the important work undertaken by community housing trusts to support the delivery of more affordable homes in rural and island communities. The delivery of what we may think of as a small number of homes can make a big difference in sustainability to a local economy and public services, and to help with the retention of people in an area, particularly young people. I have seen at first hand that high-quality energy-efficient homes are delivered by communities through the rural and island housing fund, including in Fort Augustus and Strachathrow, and I have heard from tenants how those projects have made a significant impact in our local communities. The Scottish Government has been active in identifying innovative ways to finance the pledge for 110,000 affordable homes by 2032, of which 10 per cent will be in our rural and island communities. Can the cabinet secretary confirm that this rural focus has been successfully built into the Scottish Government's housing strategy going forward? Yes, we are making £3.5 billion available through the affordable housing supply programme, this parliamentary term, to support the delivery of the ambitious target of 110,000 affordable homes by 2032. That includes support of up to £30 million through our demand-led rural and island housing fund for communities and organisations not able to access traditional affordable housing funding, and it works alongside our wider affordable housing fund. It will also shortly publish the remote rural and island housing action plan to support the delivery of homes in rural and island communities. I strongly support the community housing trust model, alongside the use of rural housing burdens. I think that it can play an important role in constituencies like mine in the East Newk, where there is a real problem with the growth of second homes and holiday lets, but not many councils know about this power or these trusts outside the Highland areas. What more can the Government do to encourage the use of the model in places such as Fife? Willie Rennie makes a fair point. I hope that the new remote rural and island plan, alongside the funding sustainability of community housing trusts, as Willie Rennie points out, is operating mainly in the north of Scotland and south of Scotland. It will be able to share some of that good practice and expand some of the good work that it is doing, because we know that it is a model that works, and I am happy to keep him appraised of that. To ask the Scottish Government what action it is taking to tackle any economic inequalities that are faced by unpaid carers. We value all carers for their contribution, which is why our carer strategy sets out what the Scottish Government is doing to tackle the economic inequalities that are faced by unpaid carers. It takes a cross-government approach to financial inclusion, including through social security and supporting carers in employment and education. The carers allowance supplement, which will be five years old this year, and the young carers grant are Scottish Government benefits only available in Scotland that are vital in this and are providing further financial support to unpaid carers. Those receiving carers allowance supplement this year will receive an extra £541 more than those in the rest of the UK. As we move into spring and summer, we are all hopeful for some needed respite from household expenditure with energy use dropping. However, that is little comfort to tens of thousands of households across Scotland who continue to exist on the precipice of financial insecurity due to exorbitant energy bills. However, carers face significantly higher energy costs as they must often operate essential life-sustaining equipment. In this chamber, in response to questions from myself and my colleague Jackie Baillie, the Government has previously stated that it would look to provide additional support for unpaid carers for that life-sustaining equipment, but no detail has yet been forthcoming. I would ask the minister if and when the Government plans to publish details and perhaps a timetable for such additional funding to be made available to carers. Thank you very much. Paul O'Kane makes a very serious point here about the impact of the cost-of-loving crisis. We have heard those same concerns from all the organisations and support organisations and carers that we know. In introducing the carers allowance supplement, it alleviates that in some way. The specific points around health and medical equipment are an issue that the health secretary is dealing with and I will ensure that he can get that response on that point to Paul O'Kane as quickly as possible. My MP colleague Wendy Chamberlain is taking a bill through the UK Parliament that would give unpaid carers the right to take leave from their job. What additional steps can the Scottish Government take to support unpaid carers and ensure that any inequalities faced by unpaid carers are addressed? Thank you very much to Beatrice Wishart for that question. I will look with interest to Wendy Chamberlain's work on that because she knows that employment law is reserved to Westminster. However, we have heard those same concerns from carers and their support organisations about the impact of the challenges around work and the rules that come alongside that. We have our fair work principles that we are working on right now. We will publish our response in the coming weeks around about carer support payment, which has just been highlighted in a recent consultation. We can come back with that information when we have that consultation analysis. To ask the Scottish Government whether it has plans to reform the compulsory purchase order process to make it easier to convert empty buildings into council housing. Local authorities already have broad compulsory purchase powers to support a range of development and regeneration projects, including bringing vacant derelict and underused land or property back into productive use. The Scottish Government remains committed to reforming and modernising the compulsory purchase system in Scotland and to make it fairer, clearer and faster for all parties. An estimated 112,300 properties in Scotland are unoccupied at any one time, with nearly 30,000 empty for over a year. Over 130,000 people are homeless or in waiting lists. Will the Scottish Government carry out an analysis of how compulsory purchase orders are currently being used with the aim of making it easier for councils to provide housing to those in need? Two things on that. First of all, an audit of empty homes is currently under way to gather evidence about the effectiveness of our current approach and help with thinking on options for future policy and funding. I should say that, through the Scottish Empty Homes Partnership, it has already helped to return more than 8,000 homes back into use as warm, safe and secure housing for those who need it. We are going to be, as a first step, establishing an advisory board that will be appointed to help inform the development of options for reform of the compulsory purchase order process. It is important that we look at what levers we have and what additional levers we need, because I absolutely agree that we need to incentivise the use of empty homes being brought back into use and perhaps disincentives to people who are hanging on to empty homes that are not used productively. I am happy to keep the member updated. Compulsory sale orders would be another lever that local authorities could use to get those properties back into use, and the Government consulted on the introduction. I wonder if it is still the Government's policy to support that introduction and what the Government has done to work through some of the human rights issues that the Cabinet Secretary mentioned last time in the chamber. The member is aware, as I am, of the ECHR implications for compulsory sales orders, and they continue to receive careful consideration. I would be happy to update Mark Griffin on that at a point at which there is more information available if that would find that helpful. To ask the Scottish Government whether it will provide an update on progress with its tackling child poverty delivery plan, including in relation to expanding free-skill meal provision. We are focused on delivery of the wide-ranging actions in best at bright futures and will lay a statutory annual progress report before Parliament by the end of June 2023. In relation to free-skill meal provision, Scotland already has the most generous provision of free-skill meals anywhere in the UK, with 362,000 pupils benefitting from support during term time, saving parents £400 per eligible child per year. That includes all pupils in primaries 1 to 5 and eligible pupils in primary 6 to S6. Recent additional investment announced means that we will expand free-skill meals for all primary 6 and 7 pupils in receipt of the Scottish child payment. I am grateful to the cabinet secretary for that update and welcome the Government's commitment to the roll-out of universal free-skill meals. Six months ago, the Government said that it would pilot universal free-skill meals in secondary schools. Can the cabinet secretary confirm which schools are involved and how the pilot is going? Will she or relevant minister agree to meet me and campaigners, including the Scottish Youth Parliament and the SUC Women's Committee, to see how we can speed that up? I am happy to make sure that Monica Lennon is furnished with the information about which schools I do not have that information in front of me. I am sure that she can have a meeting with the relevant minister in order to discuss those issues in more detail. The IFS has noted that, among the poorest 30 per cent of households, those with children will see their incomes boosted by around £2,000 a year on average as a result of Scottish Government benefit policy and progressive taxation. Can I ask the cabinet secretary for her response to the analysis? I certainly welcome the IFS analysis. It shows the positive impact that progressive choices are having for low-income families, even within our limited powers and budget. Of course, an important part of that is the 13 benefits, seven of which are only available in Scotland, including the game-changing Scottish child payment. That is in marked contrast to what is available in the rest of the UK and particularly the UK Government's welfare system. That shows the difference between our two Governments. We want to tackle poverty and inequality, not push people further into hardship. To ask the Scottish Government what its assessment is of any potential impact that the UK Government's proposed illegal migration bill could have on devolved social justice powers and the duties of Scottish local authorities. We have written to the UK Government to state unequivocally that the Scottish Minister does not support the illegal migration bill. It is imperative that safe and legal routes exist for people seeking safety and protection from war and persecution. The bill will remove the right to seek refugee protection and, of course, risks pushing people into exploitation and destitution. We should be upholding the UN Refugee Convention, not undermining its international protection. We are considering the detail of the bill, including the impact on people and services, as well as considering any legislative consent implications. The immigration law practitioner's association says that the clauses of the bill dealing with modern slavery and trafficking breach the UK's obligations to victims of trafficking under article 4 of the European Convention on Human Rights. I ask the cabinet secretary if she agrees with me that not only are the actions of the UK Government deeply immoral but they are also illegal. The Home Secretary, of course, was unable to state that the bill's provisions were compatible with the European Convention on Human Rights, which tells us all we need to know about the support legislation. Of course, it follows on from the shameful nationality and borders act and a further restriction on the provision of support to human trafficking victims penalises particularly vulnerable people. People who have suffered unimaginable trauma, including sexual exploitation, are being forced through violence to work for no pay and they will be prevented from accessing the safety and support available to them in the UK. Importantly, children will be left in a shocking position until they turn 18, where they will then be detained and removed to a third country where they have no connections or family. That is shocking and cannot be allowed to happen now. I would have thought that the whole chamber would agree with that. The UK Government's illegal migration bill is likely to have a significant impact on migrants and asylum seekers in Scotland. Scotland's legal profession alone will likely have their caseload skyrocket under the provisions for removal in the bill. Many asylum seekers that have arrived in the UK and Scotland have risked their lives and arrived with nothing. It is likely that any legal assistance that many asylum seekers will require would have to be pro bono. Will the cabinet secretary advise what the potential impact of the bill will be on Scotland's legal aid service? Foisal Chowdhury raises one issue of which there are many about the impact of the legislation. The real concern is that it leaves more people destitute without recourse to public funds. That has a major impact on services as well as the impact to the individuals themselves. We will continue to do what we can by supporting third sector organisations, some of which are involved with the issues of legal support and legal advice. We will look to see what more we can do, but let's be under no illusion that the legislation will have a profound impact on those affected and on services in Scotland. We should all unite in condemnation of the UK Government over this move. To ask the Scottish Government what action it is taking to support local authorities to ensure the provision of local services, including swimming pools and leisure centres, and meeting the needs of local communities. Minister Ben Macpherson. The Scottish Government believes that everyone should have access to local sport and leisure facilities that help to support the physical and mental health of the nation. We understand the challenging financial circumstances faced by local authorities, significantly due to the cost of living crisis. For its part, the Scottish Government has increased the resources available to local government by over £793 million in the financial year 2023-24 at a real-terms increase of £376 million, or 3 per cent. Minister, swimming pool is much loved and a hugely valued community resource, but it is due to close on 16 April because of devastating cuts to sport Aberdeen's budget by Aberdeen City Council. It is the only pool in Aberdeen to have a shallow stair entry for the elderly and disabled. It is used twice weekly by additional support needs department of Bucksburn Academy, with the nearest pool two bus rides away, and more than 8,400 people have signed a petition to keep it open. Will the Scottish Government intervene with Aberdeen City Council and sport Aberdeen to give hope to the local community and save Bucksburn swimming pool? We are aware of the decision taken by Aberdeen City Council to close the Bucksburn swimming pool and understand the disappointment that members of the community feel at this closure. However, the Scottish Government understands that, while everyone should have access to local leisure facilities, it is for local authorities to manage their own budgets and to allocate the total financial resources available to them, including on leisure facilities. On the basis of local needs and priorities, we note the member's points and, following the financial package announced yesterday by the UK Government to support swimming pools in England, we are examining what support can be provided to the sector in Scotland if the member wishes to submit any further follow-up correspondence to this question to, first of all, the local authority, but also to the Scottish ministers, we will of course look at that. I welcome the £13.5 billion local government settlement. I wonder if the minister can say any more about the new deal for local government, especially around the area of fiscal flexibility. The Scottish Government is committed to working with COSLA and SOLIS to agree a new deal for local government with the aim of agreeing shared priorities and providing greater flexibility over local funding, with clear accountability for delivery on shared priorities and outcomes. It is very important to consider the point that those are shared. The new deal for local government reflects the desire on both sides to reset the relationship between the Scottish Government and local government. The Deputy First Minister made clear in his letter of 15 December to the COSLA president that we want to develop such a partnership agreement at pace and look forward to doing that. Thank you very much minister. That concludes portfolio questions. There will be a brief pause before we move to the next item of business to allow the front benches to change.