Ffiown awaid fram Canich, North Scotland, sef dylwch nes monstws visuol ffordd ar y rhan 40 sefydl o'r August 31st. I mean at seven sightings. I've been to Loch Ness and you do start looking and go what's that? Cos you think on that day when you're there. But you need glasses. Ylb. I didn't need glasses, I was there in the 90s. I don't need glasses. You said you wanted to. I was there in the 90s. I was there in the 90s. Mi ar hyn fyrdd wedi bethau. I was there in the 90s. They were way better now. I still don't need glasses now. I still don't need glasses now. I mean with a monster wasn't it in the 90s? It was 1992 when I was there. As the podcast chief technology expert, like here's what I think. Yeah, exactly. Come on. The fact that there has got to be, there's got to be sophisticated technology that can literally sweep that lock from one end to the other and we'll see anything probably bigger than a crisp packet that exists in there. So if it was there, I believe we'd have found it.


So, I will continue my discussion about the queuing system, you can say the features that are important in determining various aspects of a queuing system. So, L denoted our average number of people in the system, L queue was the average number of people in the queue. And then, we also had computed the average waiting time that a person will spend in the system, which includes waiting in the queue plus service time. And then, we will also talk about the waiting time in the queue, which we show also is important. So, anyway, so let me just continue the discussion about L and L queue. And so, this is as functions of rho, which is your lambda by mu, that is your lambda is the mean arrival rate and mu is the mean service rate. So, lambda by mu denotes your, in a sense utilization of the queuing system. So, now you can see from the figure, you can see from the figure that is on the screen, that as you know full utilization of the server, that means if rho is close to 1, then you see that your graph for L and L queue both are going to infinity, right. So, that means when lambda is close to mu, if rho is close to 1, that means lambda is close to mu, that is the mean arrival rate and the mean service rate are almost the same. In that case, you see the number of people in the system and the number of people in the queue, they will, the expected number will go to infinity, right. You can see the vertical, the approaching, the both the curves are approaching the vertical line. So, now this what we are saying is mean values, that means L is the average number of people in the system and L queue is the average number of people in the queue, which are going to infinity. So, it is not, see it is bad if once in a while your system has infinite people or very large people or the queue has becomes very large, but here it is saying that it is a mean behavior, that is on the average the system will have infinite people, very large people and the queue will also be very large, of course. So, that is not acceptable, finally, but anyway also what you want to say here is that now in this case, this does not happen in real life, because if the, if the big crowd, then it turns away lot of people. So, it is not that the queue and this will continue to grow. So, that does not really happen and therefore, what we are saying is that in real life, there will not be a balance, because the queue system, how many people turn away and so on. So, it will not follow the same pattern and therefore, when things become so bad, we cannot apply the same rules that we started with our assumptions and therefore, we will say that the system is not in balance. So, therefore, it cannot be measured or it cannot be analyzed by any of these models that we have written down. So, if you look at the mean length of 4 on the vertical line, then you draw horizontal line from 4, then you see that it will meet the corresponding utilization is 0.8. So, mean queue length, that means on the L Q curve. So, and so L Q is the first curve and then L is the second curve and similarly, if you look at the value of L Q equal to 5, the corresponding row value is 0.85. So, that means when you allow 4 people to wait in the queue, then the server will be ideal for 20 percent of the time, because row is 0.8. So, 0.2 is the fraction of time that the server will be idle and so 20 percent of the time the server is idle and if you allow the queue size to be 5, then it will be 15 percent idleness. So, you can see that there is definite conflict that is if you want that your server should not be idle for a very long time or even not then as we saw that the if the system is allowed to operate freely, then your queue and the people in the system and people in the queue will become very large. So, therefore, but then you do not want that to happen, because then you lose out on customers and so on and it becomes a chaos. So, there is a definite conflict between the issue, between the desire to obtain fully utilisation of a server and the desire to keep the mean queue length short. So, you can see that, so wherever there is goodwill of the customer is very important, certainly the persons who are offering the service would want to make sure that the queues do not become too big, but then where it is more important that means when the finances are important and you are having a service where they you know you cannot keep too many servers, because that means that many salaries and so on and then of course, your the servers will be idle for a long time. So, therefore, one has to balance. So, the system can of course, first of all this model has shown you that the system cannot be in balance if the arrival rate is equal to the service rate or even if the arrival rate is close to the service rate. So, the ideal ratio of arrival rate to service rate is something less than 1, that is acceptable, because your service rate must be more than the arrival rate otherwise and I mean that makes sense and so, but its specific value depends on the relative cost of idleness versus congestion. So, if for you it is very important that the system should not be congested, there should not be too many people in the system, then you will make sure that your service rate is much higher than the arrival rate and if you have limitations, you cannot provide you know more than one server, maybe let us say or the efficiency at service providing the service, then you know you will go for. So, idleness versus congestion. So, if you want the allow the server to be idle, you will not have that many people waiting in the system, because your service system will be such that your mu is much higher than the arrival rate, you will ensure that and in that case your value of LQ will be small, but if your mu is close to your lambda, then there will be congestion. So, one has to really strike a balance between you know idleness versus congestion. So, this is the whole idea. So, therefore, you see we can through these this model, we can study and decide what should be our level of service and given what is your level of you know customers arriving to the system. So, now even though we the little formula told us that your if you we have already computed average waiting time in the system and now we also know that WQ can be written as rho times W. That means the average number of people waiting in the queue is your utilization factor times the average waiting time in the system, but we will also like to compute this independently, because the distribution of WQ is also of importance and besides the WQ is of importance in hospital emergency rooms, because in a hospital emergency room the time that you are waiting to be taken to the doctor is important, because we want to cut that short. It is an emergency room and therefore, people need treatment fast. So, you would therefore, WQ is of importance and in hospital emergency rooms you would want to ensure that your WQ is not very large. And so, we want to look at it in a greater detail and also obtain the distribution of WQ. So, now of course, you can immediately write down this relationship, because if there is nobody in the system when the arrival comes for emergency treatment, then probability of the waiting time will be 0, because the patient will immediately be taken to a doctor for being treated. So, then in that case your waiting time will be 0. So, probability that WQ is 0 is P 0. That means, there is nobody in the system, which is 1 minus rho. So, now we will want to compute the distribution when a person coming to the system has to wait. That means, there is 1 or more than 1 person already in the emergency room and then the person has to wait. So, this we will compute try to obtain the distribution for WQ also. So, for n greater than 0 that means, if the any customers already present in the system, then the new arrival has to wait through an exponential service times until his or her own service begins. So, if you want to find out the probability that WQ is greater than T, where WQ is the waiting time in the Q. So, then probability WQ greater than T, so I will break it up into conditional probabilities and then add up. So, this will be probability WQ greater than T given that there are n customers and so we add up from 1 to infinity, because there has to be at least one customer to be being serviced, then only there will be a need to wait in the Q. So, this is n to 1 to infinity, probability WQ greater than T given that there are n customers in the system and this then would be written as. So, now when you write this probability then this will be P n into probability S n greater than T, because if these are the n service times then S n would be T 1 plus T 2 plus T n. So, this has to be greater than T, because the n people being serviced see you are in the you are waiting in the Q. So, then n services have to be completed and that takes because your waiting time is more than T. That means, these n services are taking more than time T and this into probability of there being n customers. So, then this will be, so I will write down for P n which is rho n into 1 minus rho and varying from 1 to infinity probability S n greater than T. Now, if I take rho outside and then I write this as rho, so this will be rho n minus 1 into 1 minus rho and this is probability S n greater than T. Now, see at n equal to 1 the value here is rho into 1 minus rho and this is probability S 1 greater than T. So, if I rewrite this as rho outside and now at n equal to 0 your P n would be just 1 minus rho, because rho n rho raise to 0 would be 1. So, this would be 1 minus rho, so same thing and here it will be S 1 greater than T. So, the same event is being written here and therefore, all subsequent ones will be the same. So, this helps you to because now you have this summation, the probability sign is missing here. So, please write this will be P in probability of S n plus 1 greater than T, I will just write it, there should have been probability. So, probability S n plus 1 greater than T, so therefore, so now you have, because the summation is from 0 to infinity, so then we can sum it up easily. And therefore, you can you know just expand this and so here this is no people, then the service time of the rho into 1 minus rho actually would be the you know 1 percent in the system, then the waiting time, this is the service time of the first person is of the one person already present in the system is more than T and so on. So, when you take out rho then this is what you can, you just expanding this expression like this, this series. And this now we know, we recognize this because when we computed the distribution for the average time in the system. So, then this was, this is nothing but and that is why we wanted the sum from 0 to infinity. So, this is probability W greater than T and we already know the distribution of W greater than T. So, we know this probability, so therefore, this is rho into e raise to minus mu 1 minus rho T. So, we have computed this probability W q greater than T, which is equal to this. Now, some of the rho is creating the problem, because I cannot call this exponential distribution, why? Because if you write the pdf of W q then this will be you know this you will write as 1 minus probability W q less greater than T. This whole thing, so then d dt of that will give you the pdf of W q. So, this will be minus rho into, because this is 0 and I will write down the, so this one is rho into e raise to minus mu 1 minus rho T substitute for this here, then d dt of this rho is outside. So, then differentiation gives me this expression, which is equal to this. So, therefore, this would not be an exponential distribution, because exponential distribution would be this rho is the extra part. But if you consider the conditional distribution, so now that is important conditional distribution of W q given that W q is greater than 0 it does have an exponential distribution, because now you will write the conditional part and this will be equal to probability W q greater than T, because when you take the intersection obviously T is positive, so W q greater than T. So, the intersection of these two becomes just this event and divided by probability W q greater than 0. See, remember this you have to write as probability W q greater than T intersection W q greater than 0 and then it will be probability W q greater than 0. This is what our formula for conditional probability is, but this is same as this of course, given that T is positive. So, this event is equivalent to this event given that T is positive and then you divide by probability W q greater than 0. So, therefore, this becomes now rho e raise to minus mu 1 minus rho T divided by rho, because probability W q greater than 0 is this is what we computed here W q is not 0. So, remember our W q probability W q equal to 0 was 1 minus rho, because if the person does not have to wait that that thing there is no person in the system and so therefore, P 0 this probability is equal to P 0 which is 1 minus rho. So, if you want probability W q greater than 0 then it will be 1 minus of 1 minus rho which is equal to rho. So, probability W q greater than 0 is rho, so we divide this. Now, the rho cancels out and this is e raise to minus mu 1 minus rho T. So, this now is coming from that means, this now represents because the conditional part W q greater than T given that W q is positive. So, this is this which matches with our exponential distribution and so the conditional distribution of W q is exponential with parameter mu into 1 minus rho T. So, yeah so this is therefore, I again just want to repeat that this is not the conditional just remove this. So, it is just probability W q greater than T is that what you are computing you started writing out is this and then this is I just you know rewrote this expression and just so that I can relate this series with the series probability W greater than T that was helpful and then we saw that the distribution is not matching with the exponential. So, it is only when you take the conditional probability that you will get fine and so this already we had seen and therefore, this is rho. So, that is it. So, therefore, and this will be useful at times you may really want to know the distribution of the. So, this will be conditional distribution of W q which you can recognize otherwise the distribution of W q is given by this I mean the f q t f W q t. Now, let us compute the expected time or the average time spent in the q waiting for to be serviced. So, that we will now compute independently because this is only the conditional part. So, you will say that the expected time in the system is equal to the expected. So, the time in the system is time in the q plus service time. So, expected value of that. So, expected time in the system is expected value of time in the q plus the service time right and so this can be written as expected time of time in the q plus expected service time. So, this if we denote by W q. So, this is the convention we have been following that the variable is also treated as the denoting the expected value of that variable. So, this is expected time in the q plus expected service time is 1 by mu right because your number of services is with parameter mu exponential with parameter mu. So, 1 by mu is the expected service time. So, this is what you have and therefore, W q is W minus 1 by mu and you can. So, we know W expected value of W which is 1 upon mu minus lambda minus 1 by mu. So, this becomes what we had computed rho times W. So, W q is equal to rho times W and which we have already seen through using the little formula. So, we saw that the conditional distribution of W q and W are both same exponential and the parameter was yeah I mean the same parameter. So, they and now also I want to make an observation that the little formula that we obtained were under special conditions, but it turns out. So, that means the relationship between L and L q W and W q and L and W. So, they are all in fact what it means is that if you can find any one of the quantities L L L q W W q then you can find all the other three. So, all the four are related and it turns out fortunately that under very general conditions these formulae are valid. So, therefore, you know computing any one of them would help you to get the values of the others. Now, I want to continue this discussion on these quantities L L q W and then also show you how we then through these analyze these weighting weighting systems or with a qing model. So, this particular example is a small one, but anyway this is some Sheldon Ross and what it says is that machinery in a factory break down at an exponential rate of 6 per hour. So, that means the arrival of the machinery for repair is 6 per hour and then there is a single repair man who fixes machines at an exponential rate of 8 per hour. So, the arrival and the service rate is all provided to you. The cost incurred in lost production when machines are out of service. See the machines come for repair and they are waiting. So, the repair man has to repair them. So, while the machines are out of service they incur a cost because there is lost production. So, rupees 100 per hour per machine is lost to the organization. So, the cost of lost production is rupees 100 per hour per machine. This is what is given to us. Now, we want to find out what is the average cost rate incurred due to failed machines. So, while this is machines are waiting to be repaired, they are not producing and therefore, there is a loss to the organization and this is the rate. So, you want to find out the average cost rate. Now, the average cost rate will be dependent on the number of machines which are in the system which are either waiting to be repaired or which are being repaired. So, that will be our number L. So, therefore, you see L lambda is 6, mu is 8. So, therefore, average cost rate we will write as rupees 100 into average number of broken machines which is either waiting to be repaired or they are waiting or they are being repaired. So, this will be rupees 100 into L. L is the average number of broken machines which are in the system. L gives you the average number of people or customers in the system. So, therefore, this is rupees 100 into lambda upon mu minus lambda which is 100 into 6 upon 8 minus 6. So, rupees 300 per hour. So, the loss. So, therefore, this is a very important parameter because the system would now like to evaluate whether the repairman that they have is good enough or they need to have more repairman because it depends on what is the how much the loss to the system compared to the salary of a repairman and so on. So, that question you see will always be running through all these examples and you want to analyze this. So, this should hopefully help you in your decision process. Let us take another example. So, you have now a pump station, a single pump station. So, a single pump petrol station. So, there is only one pump and so cars that come for taking the petrol have to wait in the queue. If one is being serviced, once the serviced car is done with then the other will come from which is waiting in the queue. Now, inter arrival times are exponential with mean 12 minutes and that means the inter arrival time the average time between two arrivals is 12 minutes and the service time is exponential again with mean service time 6 minutes. So, average time it takes to fill up a car is 6 minutes and waiting space is unlimited. So, I have not put any question here, but as we go along we will see what are the kind of questions we want to answer here. So, let us see lambda therefore that means the arrival rate is 5 per hour because arrival time inter arrival time with mean is 12. So, therefore the number of arrivals per hour is 5 and similarly the service rate mu will be 1 by 6 into 60 because this is per minute 6 this is this is all in minutes. So, you convert to hour. So, this will be 10 per hour that means the service rate you can on the average you can fill up 10 cars in an hour. So, rho the utilization factor or the traffic intensity we have lot of names for this. So, rho is 5 by 10 5 is your arrival rate and 10 is your service. So, 5 by 10 therefore this is 0.5. So, the traffic intensity is not very high 0.5 is not considered to be very at means the petrol station is not very busy right now with the kind of service rate and the kind of arrival rate. Then if you want to look at the probability that there is no car at the petrol station. So, that will be 0.5 1 minus rho which is 0.5 again which is a high probability then P n is the n n cars at the station. So, that will be 0.5 into 0.5 raise to n and then the mean number of cars at the station mean number of cars would be lambda upon mu minus lambda or well this is why am I writing this as how in fact this is sorry this is rho upon 1 minus rho. So, the mean number of cars at the station is 0.5 upon 1 minus 0.5 which is 1 and l q the car waiting in the q is 0.5. So, this is the idea. So, now we want to analyze the system through these quantities that we have computed and you know like you want to again answer the question that in case the arrival rate increases then supposedly the traffic intensity will go up because this number will go up and then what kind of numbers l and l q will be there the values will also go up and therefore, the petrol or the station owner may want to ask a question as to should we install a faster pump and so on or of course, more than one pump you will the system that model we will discuss later on when you have more than one server right now we are talking about only one server systems and therefore, the only option that the man may have in case the arrival rate goes up in the other option would be to install a pump which is you know filling up cars at a higher rate. So, we will just look at the analysis with the numbers. So, through this example which we are just discussing I again want to raise the issues about you know validity of a model. So, it is very important that you keep recalling what are the assumptions under which we are working and what are the. So, for example here I wrote that waiting space is unlimited, but you know that in a petrol station waiting space cannot be unlimited and usually you have space for 2 to 3 cars.  So, therefore, that right now it is not an issue whether the space is limited or unlimited, but in case your data changes then to say that your waiting space is unlimited is not a very valid assumption. So, therefore, one should always keep this in mind that the better model would be when you talk of queues with limited capacity or with limited waiting time. So, that would be a better model for such an example then the arrival pattern is state dependent. Now, here to assume that lambda will remain the same all the time is not correct because if there you know already 2 to 3 cars waiting the person may want to go to the next petrol station. So, therefore, this is not and arrival process is not stationary also. So, it is state dependent and also not stationary because during the rush hour there may be the lambda may be higher then corresponding to when it is a so therefore, the lambda itself may change during the day. So, the lambda is not stationary and it is also a state dependent because people do not like to wait for too long because as you know you can always drive fast you know further and get another petrol station. There may be other considerations also that is true, but sometimes people like to wait at a particular station because they are familiar with it people know them and they like and there are so many other reasons. Then also we must keep this in mind that whatever computations we are doing remember they are they are not giving us accurate information about L and L Q and the remaining other parameters W and W Q, but we can certainly change the values of lambda and mu and of the system. So, that means we can study the changes in the system and then correspondingly see what are the changes in these numbers L, L Q, W and W Q. So, this model certainly will help you to study the changes whatever changes take place in the system then we can accordingly find out the changes in L and L Q. So, that is what I want to so that is what I have stated here that the real use of the model is in evaluating the effect of changes in lambda and mu. So, for example, the station owner has the choice of has the alternative of installing a faster pump. If he does that then the mean service time is reduced to 4 minutes per car. It was earlier this was earlier 6 minutes per hour on the average. So, now the average time has gone down to 4 minutes. That means the patrol pump can service 15 cars per hour. So, by installing a faster pump the mean service time is reduced to 4 minutes that is the pump can fill up 15 cars per hour. So, your row would now become 5 upon 15 because the arrival rate is 5 per hour. So, the intensity traffic in sent is the intensity or as you call it the utilization of the patrol pump and so on. So, that is 5 by 15 which is 1 by 3 and this is 0.33. So, this is less than 0.5. So, the earlier one was 0.5 traffic intensity. Now, it has come down to 0.33 and probability that is no car at the station. So, that comes out to be 1 minus 0.33 because it will be 1 minus row and so that is 0.67 which is greater than 0.5. So, that means the patrol pump would be vacant for more time. The fraction of time will be higher here than here because there it was 0.5. Since row is 0.5 so 1 minus row is also 0.5 and your L the average number of people in the system or at the patrol pump that means the number of cars getting filled up or waiting to be filled up. So, that will come out to be 0.33 by 0.67 which is also less than 0.5. So, therefore, this does not warrant installing a faster pump because your patrol pump is vacant longer the intensity traffic intensity has come down and so on. So, if you are looking at it from the view point of the patrol pump owner then certainly it does not warrant installing a faster pump. In case the arrival rate goes up that means you have 6 cars per hour instead of 5 cars per hour and with the current pump that you let the man has then this will be row will be 6 by 10. So, this will be 0.6 that means your traffic intensity will go up from 0.5 to 0.6. Your probability of there being no car in the system is 0.4 and your L is 1.5 and your L Q is 0.9. So, therefore, you see that is what I am saying that now with the model you can play around and for different values of lambda you can figure out what is how these numbers are changing and then if we can as I said you know the losing the good will of the customer versus the cost of installing a faster pump and so on. One can the owner can study all those things through this model. And therefore, that the basic contribution of this model lies in being able to study the various changes that will take place in your you know you can call them parameters when you change your lambda and new change. So, now I will after having discussed the one server model we will now look at the situation when there is more than one server. So, this is the model would be MMS that the same the pattern arrival pattern and the service pattern are the same. You can say that and therefore, the number of servers is now more than 1. So, in that case you see as long as there are number of people is less than s then your service rate will be because whoever is there in the system if there are n people in the system all will be serviced. And therefore, your service rate will become n times new that is understandable because everybody is being serviced. And therefore, the n people who are being serviced simultaneously. So, the service rate you can say has gone up to n mu and if you have more than n s people in the system then of course, it will remain at s mu because only s people can be serviced since you have s servers. So, therefore, then the service rate will be s mu, but I will try to explain. And therefore, diagrammatically if you look at the transition diagram here then you see arrival rate is the same lambda which is you know. So, lambda is the arrival rate, but the service rate changes. So, if you have 1 person in the system then it will be mu because you serviced and therefore, you get back to 0 state. If you have 2 people then 2 are being serviced simultaneously and therefore, you have the rate at which the system can transition from 2 to 1 will be 2 mu. So, this should be understood very well because so what we are saying is that many people are being serviced and therefore, the probability of transitioning that means, the rate at which you can transition from 2 to 1 will be 2 mu. And similarly, with 3 people the rate of transition will be 3 mu, but the arrival pattern is the same and therefore, the arrival rate is lambda. So, this will go on up to s minus 2 and then up to s minus 1 and when you have s people in the system if your state is the system is occupying state s, then it will be s mu and thereafter the service rate will remain at s mu. The arrival pattern would be at the rate of lambda, but the service rate would then remain at s mu. So, this is the idea and so here is the service rate depends on the number of people in the system that is if the number of people is less than s then it is n mu and if it is more than s then it is s mu. Now, let us understand the assumptions which is very important and what we are assuming is that all operators operate at the same mean rate. So, right now this is a simplification because obviously, it will get very complicated if I have different servers with different service rates. So, therefore, we are assuming that all operators operate at the same mean rate mu. Therefore, I am saying that when there are n people in the system the service rate will be n mu and when they are more than n it will be s mu. So, this is possible only if I make the assumption that all operators or all service people are operating they have the same mean rate mu. So, this makes it because then we do not have to keep track of which servers are busy you know because then we will have to accordingly keep changing the service rate and that will become quite problematic right. And then again I feel that this is not very this this can be treated as a realistic assumption because you know a person may be a little more efficient than the other but the differences cannot be very very large to really take care of them in the system right in the model. So, this is what one assumption and then the second assumption which is important is that departures will be one at a time that is probability of service of two or more services being completed exactly at the same time is 0. So, this probability that means the departures because that is important of the you know when we say that we are looking at M M S system and yes I will have occasion to explain you know through the when we talk of Markov processes we will discuss in detail. And so anyway and when we are talking of Poisson process remember I had told you that there is a always a small interval there is a small enough interval in which we say that the probability of one arrival is probably you know there is something like lambda delta t right and then for more than more than one arrival it was of the order delta t square and higher order right. So, therefore we were neglecting the so the probability was again that means we assume that exactly at the same time two or more customers will not leave the system. So, the service will not get completed exactly at the same time for more than one people. So, therefore there will be a distinct interval between two departures. So, therefore we can model this as a birth death process because our birth and death process the assumption when we talking of M M S. So, then the basic assumption is that your arrivals and departures are distinct at distinct times you cannot have more than one departure or one arrival at the same time. So, once we make that assumption and of course the second assumption is that the operators are all operating at the same efficiency. So, the mean rate is same then we can process this system as a birth death process right and so now one can write down the balance equations. So, this is the first one which is easy to understand right because you have you already have one person then the rate at which it can depart is mu. So, mu p 1 must be lambda p naught one person can arrive when you have 0 when you are in a 0 state. So, then this and this gives you this. Similarly, when you want to write for so two people in the system then it will be lambda p naught you can go to p 1 sorry to 1 and from here when you have two people then at the rate 2 mu you can go to again because one departure one person gets serviced and so you again go to p 1 and here it will be lambda plus mu right p 1. That is why the transition diagram and even when I were discussing mm 1 system I had explained to you how you can you know interpret the transition diagram. So, it will be lambda plus mu. So, it is actually not there is nothing new here except that you have to remember that. So, I have not written down the remaining things because it is understood that up to s minus 2 you will have this thing and then after that you will the moment you have s people in the system s or more then this is the balance equation that you will get right. So, once you have written down these balance equations immediately you can start solving. So, p 1 in terms of p naught will be lambda by mu p naught and then if in the second one if you substitute for p 1 in terms of p naught then you get your 2 mu p 2 as. So, the expression 2 mu p 2 simplifies to equal to lambda square upon mu p naught and therefore, p 2 is equal to lambda square upon 2 mu square p naught. So, note the correction that 2 was missing. So, it should be lambda square upon 2 mu square p naught. So, all the probabilities can be computed in terms of p naught. So, important thing is that the moment you have more than one server things change a little and one has to understand under what assumptions you are now modeling the situation and so I have tried to explain to you how we will under what assumptions we will treat this as a birth death process. So, the services are all at the same rate that means all servers have the same efficiency and that no more than one departure at exactly at the same time. So, there will be distinct interval of time between any two departures from the system. So, under this you can easily write and then of course, the service rate changes depending on the number of servers you have and so under all these three assumptions you can write these balance equations and then you will try to get the probabilities in general formula and then we will again compute your quantities L, L, Q, W, W, Q to get an idea about. So, therefore, your traffic intensity also will change. You can see that because the service rate is changing and therefore, your traffic intensity will also change. So, all these again open up a very interesting situation and we would like to look at them.


The National Assembly has approved the participation of independent candidates in political activities in the country. The Red Chamber also approved that its presiding officers would henceforth be members of the national security country. The chairman, the Senate ad hoc committee on the review and Deputy Senate President Oviya Omakege, in a motion informed the Senate that the bills have been passed by the required number of states to make them form parts of the constitution of the national fifth or treasure bill number 46 membership of the National Security Council to include presiding officers of the National Assembly and bill number 58 independent candidacy 2023. Mr. President, the Senate recalled that on Tuesday, the second day of May 2023, the clerk to the National Assembly was directed to transmit Constitution of Constitution Bill number 20 with the uniform retirement age for judicial officers after it met the requirement of Section 9, Section 2 of the Constitution to Mr. President for his accent. Also recall, Mr. President, that Gumbay State House of Assembly was among the State Houses of Assembly that were yet to forward their resolution. Others are Jigawa, KB, Quara, Plateau, and Therabar State Houses of Assembly. Aware, Mr. President, that Gumbay State House of Assembly has now forwarded its resolution, convinced that with the approval of the Gumbay State House of Assembly, the Constitution fifth or treasure bill number 46 membership of National Security Council to include presiding officers of National Assembly 2023 and bill number 58 independent candidacy 2023 have now made the provisions of Section 9, Section 2 of the Constitution for passage. Accordingly, Mr. President, the Senate reserves that the Senate do direct the clerk to the National Assembly to transmit the bills, both bills, to Mr. President, the Commander in Chief of the Armed Forces of the Federation for his accent in line with the provisions of the Act's Authentication Act. After conceding the motion, the Senate directed the clerk to the National Assembly to transmit the Constitution alteration bill number 58 to President Muhammadan Buhari for sent in line with the provisions of the Authentication Act.


Welcome to the Distribution Stab Room. Our first speaker this morning is, no, don't undue talking. Our first speaker this morning is going to be Andres Shedora. Andres Shedora, who will be presenting Managing Build Infrastructure of the Debian Derivative. Yes, the talk is being recorded. The video has been set up and his microphone has been undue. It requires no human interaction. You show about this. It looks like it's only... Point another wall. It's supposed to be pointed correctly. Well, wall is also quite nice thing to see. You are in fine, sir. Thank you very much. You can hear the t-shirt over there. It speaks to the fine process of the project. It can't be listened to. Okay, so I probably, I didn't need to present myself once again. But I will do this anyway. So I'm Andres and I'm a Debian developer. And, well, I started using Debian around 2006. And very soon I understood that, well, Debian is great because of the contributors. So I started contributing myself at first by sending my reports, then patches, and then finally I became... Well, I started maintaining some packages on my own. And in 2013 I became a Debian developer and started maintaining even more things in Debian. And then two years later I started working for Collabora, an open source consultancy. And which is great in a way that we were doing lots of fun things with open source in our work time. And at Collabora I started working on a project which is called Apertis, about which I will talk today. And just some background.  Even though I've been part of a Debian project for quite some time. And so I've got no idea how to run proper things like BuildD or Deinstall and other things. And because I started working on Apertis I had to manage quite similar things, but not based on normal Debian software at all. Right, so Apertis is a Debian derivative distribution, which is based not just on Debian but on Ubuntu and a bunch of other packages we've developed ourselves. It's been originally developed for automotive purposes in particular entertainment systems in the cars. But since then the focus has been shifted to all sorts of other onboard computers in cars. And as a part of the project we provide lots of infrastructure for development based on Apertis, which means code hosting at the moment is Cegit, but we're planning to migrate to GitLab soon. Backtracking system, code review at the moment is Fabricator. And of course package build services, image generation services and automated testing based on Lava. But which I was going to talk as well at Forza, but unfortunately my talk wasn't accepted. So Apertis is mostly based on Ubuntu at the moment, so we are a derivative of it. We take a bunch of packages directly from Debian, most notably SystemD, DBS and some other things. And we've developed a few software frameworks of our own APIs and other software packages. So as I mentioned we use SystemD for process tracking and application life cycle tracking. We started using it many years ago when it was just a new thing in Ubuntu only. So this was one of the reasons we used Ubuntu because they were upstream for up armor. About a year ago I think we've migrated from Xlog to Wayland for graphics and we've been using GStreamer for multimedia playback since the beginning. In Apertis we use Flatpak in OS 3. So the application bundle format is based on Flatpak which itself is based on OS 3. It's not quite the same thing as Flatpak's bundles but we plan to migrate to actual Flatpak and not the solution based on Flatpak soon. We have an update system based on OS 3. Originally we had our own system based on beta butterfests snapshots to make sure that updates are atomic and we've got recovery modes and so on but because of certain drawbacks of butterfests we've developed our own system based on OS 3. So couple of more details. First of all the butterfests system had significant maintenance effort because we had quite a lot of custom code based around working with butterfests snapshots. Then our bootloader didn't support butterfests so we couldn't use it on Slash Boot for example and other file systems which had to be something else. With butterfests we needed to use InetromoFest at all times which isn't the best thing sometimes in a better system. And because the updated directly manipulated butterfests volumes it was unsuitable for for example unprivileged containers. And then with butterfests testing was a bit difficult and butterfests itself has quite a lot of issues which are being tackled but still like it's not I myself don't consider it a really production ready system. So in comparison OS 3 works with any file system at all any file system which supports Unix semantics. You can store multiple trees in the same repository which is definitely a benefit and you don't need extra partitions for upgrades so it saves quite some space. Because it's a more complete solution on its own we have less custom code so the maintenance is much easier. And because it's a user space solution it works better with containers and we can use it with unprivileged containers as well. So some people may ask question why are we using this crazy mix of Ubuntu and Debian and not just Debian because Debian is great universal and obviously it's free software and it's a community not a company. So despite all of those great things about Debian at the moment when Apertis was created there were certain issues with Debian. Most importantly if there are people here who are here who ran Saad for example they may remember that it took I think four years or about that to get it released because in Debian the policies the release is when the release is ready. Until recently there wasn't any particular timing employed so if there were too many release critical bugs the release might be delayed even years. Since then it's changed but when Apertis was created it was still an issue. And then stable it's a bit too stable for many people and you know with the frequency of releases it means that you get updates only once in two years and it's not always what we want. And then with unstable things may break too often and well I myself broke unstable two weeks ago when I dated one of the key bits of it which was Excel. Unfortunately I didn't do much testing, didn't do enough testing and I uploaded something which was broken for a couple of hours until I uploaded it fixed and a couple of people, well quite a few people ran into this issue and well it's been resolved since then but this is an example of what may happen in stable we cannot use it directly, you can pull all of updates from there. So Ubuntu on the other hand has also quite a large install base because there were lots of people who are too lazy to run Debian they just run Ubuntu for example and so there's quite a lot of testing despite more frequent releases. So Ubuntu is an upstream for up armor which was quite important when the apparatus was created because we heavily rely on up armor for privilege separation and security. And well they've got regular time to releases and they also had an LTS when apparatus was created so these days we've got LTS in Debian as well but back then it wasn't yet like that. So there are a couple of downsides. Obviously our apparatus is a derivative of a derivative. Oh what's going on? Oh yeah. So this complicates upgrades and rebates because you know in every release of Ubuntu there's quite a lot of updates coming from Debian originally and when we're rebating to a new version of Ubuntu it just this impact amplifies. And then we don't really need all of the things we have on Ubuntu like mayor for example and other things so it doesn't add any value for us because we use Wayland we don't even use Xorg anymore. I think there's a slight missing. Anyway so we are currently reconsidering this decision and we might change it back to Debian because lots of issues Debian had when the decision was taken are no longer issues and it might make sense to rebate to Debian LTS for example. Right so now a bit more about the infrastructure we have. So the core of it all is open build service which builds packages in controlled environments and maintains repositories from which we create images and the updates and so on. So this code hosting and review tool at the moment it's a combination of Cigit and Fabricator but we will be using GitLab quite soon for code hosting and probably for reviews as well. So we pull a few packages directly into our Git like systemd and debas so that we can apply our patches to them if needed or we can use newer versions from Debian and the rest goes like the normal packages from Ubuntu Debian they go directly into OBS but I'll tell you more about what OBS is and how we use it in a moment. So OBS every time it needs to build a package it creates a fresh route of operatives in this case builds a package there and publishes the binary packages the devs in internal repository. So all the source packages are version controlled. The system resembles subversion in a way because it doesn't handle proper branches like in Git you can create branches but it's basically just a copy with some version information which you can build packages in their own space they are not published in the normal repositories so you can test how things work in the branch and it has quite some fine-grained access control tools so you can let your users some users can just read packages, some can commit, some can do reviews and so on but with more information OBS was in the talk of Android yesterday you can watch it online. So I purchased a split in multiple components which are target, development, helper lives and so on important is snapshots. Each of them is an OBS project or so on so it's more or less independent from others in OBS project can use each other for builds. So for example here development can depend on packages from target and SDK can depend on packages from development and this means that if the packages in development require dependencies to build from target they are installed in the truth by OBS and used but normally not other way around so an exception of this is basically target because target needs some packages in development to build but basically this split is mostly for images we have separate target images and we don't want development stuff like GCC in them so while GCC can be used for builds it's not installed in the images so it naturally goes to development not target but Linux kernel is obviously in the target so when packages get built they are published in internal repositories in OBS which are not in apt formats but they use for OBS itself to build other things basically so when your package is dependent on something else OBS internally fetches the their package and pre-installs it in the truth but for apt we use reprepro which takes packages from OBS internal repositories and publishes them in format apt can understand so in OBS it's quite easy to do full distribution rebuilds by just adding one more repository to the project OBS starts rebuilding all packages in this repository and they go nowhere they stay in the repository but they are not published in the actual apt repositories until you explicitly ask OBS to do this and well it's quite a tiny screenshot on the screen but basically you can add as many repositories in OBS you like and you can select like all like you can tell OBS which other package projects it depends on what architectures to build for as you can see we don't support 586 for example or ARMv6 so as I mentioned the packages to which we apply no changes relatively to Ubuntu or Debian they go direct to OBS we just import them with no changes some packages where the patches are trivial like you know fixing minor tiny things we maintain them directly in the OBS using the OBS version control system and we just apply patches top of them in the Debian source package and increment the versions so that we can see where we have local changes and we use the fork of Ubuntu's Mergematic tool which pulls new updates from Ubuntu and does automatic merges when it's possible and then packages where this approach doesn't work we keep them in Git and use Git for merges and other things so that we don't rely on Mergematic at all those packages like I mentioned SystemD, Debas and Mesa and so on so for non-operative packages packages from Debian and Ubuntu we use the 14 format from Debian so we have upstream and apertest namespaces for tags and branches obviously we try to reuse the Git repositories from Debian so we also have Debian slash something tags and branches in repositories normally and we use Gitvill package and GBP PQ to manage the patches in the repository you can read about them in one pages for Gitvill package basically I won't go into details and for native apertest package we have a similar scheme so the master is the version in the current release and there are branches for previous releases like 17.12 and we normally it's an approach which is not accepted in Debian usually so we keep the Debian metadata the package and stuff in the master not in the separate branch as normally is done in Debian because it just adds some maintenance burden for no reason and we normally just put tags on those branches for the releases the version numbers are normally like 0 and then the release number and the just increment so Jenkins builds every commit on those branches in a controlled environment and if the build exceeds it commits the result to OBS and OBS creates major quests and builds it in the proper truth which we then review and merge we use a tool called build snapshot written by Simon which just creates releases out of Git branches and if it's work in progress not tagged as a proper release it just builds a package which has this funny Git revision number in the version so that we can see its development version and every time new packages are submitted at Fabricator they also get applied and build by Jenkins so until recently we used linear image tools to build images it was split in multiple stages I will talk about this later but the linear image tools proved to be quite difficult to use and maintain so we've changed it to our custom tool called DevOps which does the process much better and reliable so this was the original scheme we used so we had this separation between hardware dependent and independent parts so OSPAC was basically a table with the root file system not including any hardware dependent things and hardware part was the kernel, firmware and other things and we would combine them into images like this so with DevOps DevOps is a tool written in Go which is basically a Debin OS image builder with DevOps we split the build process into stages in a YAML file and every stage does its own thing and you can use templates similar to Ginger 2 basically and select parts of it depending on the type of the image it's a really cool tool written by Shorz, Simons and Denise Pinkin basically and here you can read more about it so in this tool we've got rid of the hardware parts as such we still use the term but it's actually just packages and files installed into the images when they're built we also generate SysRoot for a software development kit and then the Jenkins jobs they trigger tests in Lava and when the images are built it also goes through the fabricator and closes the bugs which were fixed in the images so this is how it looks in Jenkins when it's all green and happy so there are quite some challenges working with all these because obviously a merge-o-matic cannot handle anything more complicated than just a couple of patches applied locally Git does help but we cannot put all things in Git so we need to decide what can we do what can we put in Git and what have we maintained manually removing all packages is quite tricky sometimes because you need to check what's gone in Ubuntu and Debian and so on and draw a couple more issues with OBS basically because OBS is not Debian stuff and it doesn't do the same things with building well as I mentioned we're going to migrate to GitLab soon and become a common platform for automotive systems and not just infotainment and to learn more go to aperty.org and there's a wiki where you can read those things and learn more about the system this is it


Okay, so hello everyone. We're going to break the barriers of technology today and actually do an open stock presentation over Skype So my name is Iran. I'm from IBM Research and with us over Skype is Alberto Messina Alberto Yeah, hello everyone. I Have to apologize because I would have very much like to be with you there in Paris But unfortunately my son decided to pass me a virus and so this happens with small children I'll do this via Skype Okay, so we're going to talk about Docker Midswift the broadcaster's experience What we're going to talk actually about is Alberto to the magic. Yeah Yes So what we're actually going to talk about is how to move the media workflow Near the storage or near or near where the data is so we're gonna first Start by saying what that means Then Alberto would say would would elaborate on why we want to do that and give a demo and then I'll go and Try to look under the hood of what actually happened there. So moving the data near the storage basically means Moving I'm sorry moving the workflow of the media the media workflow Near the data actually means co-locating compute inside storage And to do that we're going to use what we call a Docker powered Swift cluster So an important thing to say here is that when you are going to run untrusted code from External user inside your storage system. You want it to be sandbox, right? And by Docker powered Swift cluster. We're actually using Linux containers to do the sandboxing And as I'll show next we also of course makes use of the Docker functionality to distribute and to Actually manipulate images, etc So Alberto if you can move us to the next slide, please So just few terminology items before I hand it over to Alberto who would talk. Why do you want to do that? etc, so Here are some Terms we're gonna use a store it so a store is essentially the computation That runs inside the Swift cluster Think about a piece of code that does filtering Transforming or whatever and this toilet as I've mentioned is going to be executed inside a Docker container as we want sandboxing Stoll it engine stall it engine is the software that invokes a storage Which runs inside a Docker image and it basically takes care of connecting all the input and outputs to the compute Then in the demo, we will also mention metadata search with this which is a Swift extension That basically provides the ability to look for objects according to their Swift metadata So with that said I'm transforming over to Alberto who will talk about this from the media side Yeah, thank you very much Iran so yeah, what does all this mean in the current Media domain, so we are actually facing a couple of challenges recently Maybe not all of you know that right is the Italian public broadcaster, but so These these are some of the things that we are currently investigating not only as our research, but also From the say from the industrial perspective So first of all what we are seeing more and more is the increase of content quality this means not only a resolution so not only Switching from standard definition to high definition to ultra high definition Also frame rate and dynamic range Which are also parameters that increase the quality quite a bit if you have experienced those The other thing that we are Facing is the increase of distribution and publication channels They for example digital terrestrial satellite, which are say okay commonplace but internet and mobile are more and more becoming of The same importance for media companies as the traditional broadcasting perspective Together with this there is another key element, which is the restoration and digitization of archives And this means millions of hours really mediums so this this of course introduces a lot of Problems related to the how to store how to manipulate how to process all these digital media In this context management of media is becoming of course critical for at least three aspects first one is quality assurance and enforcement so being able also to Enforce that a certain quality level is met Then second content accessibility, so we are talking about millions of hours So we have to be able to access this content rapidly and Last but not least definitely is the storage and processing capability that has to grow at the same pace So let me give an example of this which is rather simple But at the same time very important loudness what is loudness first of all is the perceived oral energy of an audio signal It is different from several other ways to measure Audio energy like for example RMS Because takes into account psychopsysiological aspects so on the right side of this slide you see the the ISO 226 curves that Say illustrate how the human hearing system is sensible to different frequencies with different With different magnitude. So this means that loudness of an audio signal has to be measured taking into account this So this is a very important quality parameter for broadcasting because of course, you don't want your audience to be shocked by sudden change of loudness level in both directions So around this topic a lot of work has been done from the normative perspective And so a lot of specification exists in in this domain one of the most prominent Standardization body in these cases ITU but together with ITU also the EBU which is the European Broadcasting Union also provided quite a list of technical provisions to make industry and users aware how to measure and how to assess whether audio signals are in the admitted range for a Certain level of quality So loudness is a very important thing. But what is the problem then? I mean, it is a very simple to understand is just measuring some energy of an audio signals in some In some way the program is that Programs are not normalized to the admitted level of knowledge. So you can really get whatever level This this is because of several reasons for the fact simply for the fact that programs and media are produced by different parties So these apply Different criteria. So these loudness measurement and normalization is not yet. Let's say Accepted worldwide So this may cause very serious quality problems at the client side because if you are if you are someone who's hearing to a television program and Suddenly these changes to Advertisement spot and then if the loudness is not normalized and aligned You yourself would have experienced some some let's say annoying effect And this is something that we don't want. We want to have let's say a Normalize a uniform loudness measurement and loudness level for our broadcasting channels This of course also implies Issues in the media production because you have to check the conformance of these files when they are ingested when you for example purchase new content or when you digitize your archive and You have also to check conformance after editing because if you edit content with a for example with an editing Machine, you don't have 100% assurance that the loudness is treated appropriately. So you have to check also That loudness is okay after Editing so To do this as it is illustrated briefly on the right side of this slide. There is a specific Meter that is called LUFS a loudness unit full scale, which is the The unit with which we measure loudness and we have to uniform the loudness the integrated loudness value to this to this To this parameter to be sure that the quality is okay. So then I say which is the the scenario the crowd the typical Technical scenario in which we are is a lot of file staging. So file staging is basically having different sources of of of material of digitization trading and purchase or international feeds and Everything goes into a file stage in area where files are are put for Processing for being processed and this includes quality checks metadata extraction and trans coding So in the demo that I'm going to show there will be three steps So the first will be about filing just the second file check and the third one loudness Renormalization so this will be around these these use case and we are developing this this demo we have used The systems core functionalities as Iran has introduced a few minutes ago so the solids and the ability to find objects based on metadata so let me switch to the To the to the console what we have seen here is the console of the system. So I I log in with with my colleague username and password and What I see as a first screen is a list of what we call projects So a project for that for us is the unit of production So a project could be an editing project or trans coding project if you want to transcode the media or simple ingestion project behind each of these projects a Count a container in the swift Backend exists. So we use containers the swift containers to implement the projects. So let's let's open one of these What happens is that for example, there are already some files here You see that the extension of this file is MXF. So MXF is the material exchange format Which is the professional format for media exchange in this in this In this dashboard, I can do everything with these files including for example Searching by metadata deleting stuff and so on. Let's let's for example Delete a couple of a couple of items. I do this because I want to reinsert the same items to the demo Okay, so what I I can do is for example, I'll pull the new new files Let's for example use these five seconds MXF file Which is somewhere on the network and I want to upload this file because for example someone has As both in this this this program in my company And so I have to check if this program is is correct from the loudness point of view. So I Upload these this file. Let's hope that this is working. Okay. Yes after a few seconds This is on so we are talking here of a data rate of 50 megabits per second. So the master quality for production is 50 megabits per second So something which is quite unusual in the let's say normal internet distribution of content As you have seen the upload of these files be quite quick So this means that the system is as a very good network. For example, so let's refresh the Situation. Okay, here it is. So if I go here What happens is that while I'm putting this file on the in the container already a first couple of storelets Have run the first one is a store that which extracted the technical metadata of the file So something like expert ratio the number of audio channels The depth of the components a lot of other parameters like height widths and so on so these parameters are being extracted by a storelet at the same time that the content of Was being uploaded in the container. So there was a storelet which was reading the stream the incoming stream Calculating the metadata and finally writing the file and the metadata in the container As you see here, we have a value of loudness that is Integrated loudness which value is minus 20. So the normative value should be minus 23 So this means that this file has an extra 3db of loudness which can be they say critical Because if I if I edit this file together with other content, then I may I may feel a mismatch I may feel a step which could be annoying in the in the loudness is the lowest Level of this file. So I have to normalize this in this in some way So imagine that you don't have just one file, but we have maybe 100 200 files So the first thing to do is being able to search all the files that have that problem So I search for integrated loudness Float greater than for example minus 23 Okay, so what happens? Okay, I find I find two objects in this case I used the metadata base search of the swift Backend and so I found these two files This one who has been uploaded Before and and this one which has been just uploaded. So I open this So I look again at the metadata just to check. Okay, it is minus 20 the same as before So what I do? I can do an action here to normalize the loudness of this file I launched this and what in the back end what is happening another store that which is the Loudness normalization stored as downloaded the object from the back end swift calculated and renormalized the audio level and saved a new file which with With the right longest parameter. So this should be If this has worked, I should be a scene in the metadata list that now the integrated loudness is Minus 23 so actually the file has been normalized. I Had also a nice way to To show this maybe maybe Not all the audience will notice the difference, but if I have if I play the file before normalization So we can't hear that but if there any indicator we should be looking at Yeah, this should be the indicator should be the These bars here you see these bars Yep basically this Okay, so if I open the normalize one these bars should be normally lower Okay, normally maybe This is not really This is not really an exceptional level because it's just 3db But but there are cases in which this difference is much higher and we can find files for example with a minus 10 Loudness level, which is really really loud. So we have really to To adjust this otherwise we have for us. So this was basically the demo So I can switch back to the presentation give the for back to Iran to explore the details of what I've been Demonstrating. Okay. Thank you a better. I'll I'll I'll need your help clicking So let's move to the next I think oh, okay, right so I'll first try to explain what is a docker powered swift cluster. So I'll need a click here So a docker powered swift cluster is first of all a swift cluster. We can see the proxy layer We can see the storage there then it is augmented with click With docker and also we have a docker registry private registry. I'll show next how do we use that and then another click This is like a now a year of work is is appearing in a click now We have two pieces of middleware, which is basically the storage engine We have a middleware on the proxy layer and then a different middleware on the object servers You can you can tell the difference by the colors Another click Okay so so once again a Docker powered swift cluster is a swift cluster with docker and our pieces of middleware So far we've been talking about running stolets inside the swift cluster Namely, we're bringing code from the outside to be running inside docker containers running on the store storage system However, a very basic functionality here would be to allow the user also control The underlying image where the store it runs so in our case for example, there has been a Usage of ffmpeg to actually code those toilets So what I'm going to talk about now is how does the flow of the user? Updating the image where his toilets are going to run is working. So Imagine that we have this setup we've got Docker deployed with With images that currently consist only of Ubuntu 14.04 and the stolet stuff We don't have the ffmpeg yet there. These are the small squares that are Drown here So this is the basic image that we provide and we have an image pair swift account, okay so now we want to the Swift account manager, which is basically the customer of the provider wants to To add ffmpeg to that. So first thing he does he does had docker pull out of the registry He gets the default image, which is basically Ubuntu 14.04 and our stolet stuff The stuff that needs to run in the docker site in the docker container site and then using a standard Docker tools docker file for those of you who are familiar with that. He adds ffmpeg on top of it then a click would actually demonstrate a Docker push to the repository and Then the swift stolet manager, which on the provider side needs to deploy that to the swift nodes So what's nice about docker here is that if we already have the base images deployed We only need to deploy the layer of ffmpeg. This is one of the nice feature we get from from docker So once the ffmpeg is there we can actually go and run stolets that uses it inside it So let's move on to the next slide and talk a little bit about How to write a stolet and how to deploy it so writing a stolet is extremely easy you basically need to Inherit from an interface. We currently support Java based toilets, but it's fairly easy to add Any other language binding see Python or Fortran might be problematic Once if we if we have a quick look at that that API we see that we've got in streams This is where the data would stream into the stolets. We have out stream This is the streams where the stolets could Push back or send back the computation results. We have parameters, of course, and we have a logger So once we've got this written The user can the developer can pack it into a jar. Let's have a click there and And basically upload it as Regular swift object into a designated container. Here's the put Here's how the put looks like so we do an HTTP put To a container named stall it under the account 111 in this case and this is the name of the the store it mxf technical and the extraction Extractor store it this is the store that was used in the demo during the ingest mode Where during the upload it extracted the mxf functional features and place them as a swift metadata Okay, so we and from there When the store it will be asked to be executed on a certain node or middle over fetch it from there and will Execute it inside the docker container. We'll see that in a minute. So let's move on to the next part So this is running and deploying a store. So now I'm going to talk about how to invoke stallets So there are three ways. I'm talking. This is the first way that I'm going to talk about is during put again in the demo What we've seen is that During the put of the mxf files. We've extracted those metadata features and put them as swift and swift metadata the general case would be That we don't that we want to upload some data But this is not the data that we want to actually save inside the storage system But rather we want to save a transformation of it. Okay, this would be the general case for invoking a stall at input or to enrich It's metadata, right? Think of encryption compression Transcoding whatever that you want to do before you save the you save the data as an object in swift so Telling the system that you want to store the store to run There is extremely simple simply add this header to the put request Saying extra on stall it and the name of the stall it as it was uploaded in the previous slide so Let's do if I Albert would click would help now Okay, so now we're zooming into into the proxy server, right? So we've done a put the put hit the proxy server and now we're inside the proxy server on the left-hand side We see our middleware on the right-hand side. We see the docker with the image Per the account that the request was made to so first of all we intercept the request then a click Where What basically happens now with you that we establish a connection with the actual docker container for that account One thing to say here is that on the first When we first For the when we run the stall it for the first time on that node We're gonna invoke or spawn a demon inside that docker Container that will wait for additional requests. So we don't need to Spawn new container or spawn new processes where new requests are coming. We're doing it only for the first time Alright, so we've invoked the demon. We've made a connection. We now pass the input and output of these So the point to make here is that the docker container is completely isolated No block devices no network devices nothing all the data is being streamed in and out by those file descriptor that we pass Here from our middleware to the container. Let's click on So the stall it does whatever he does in our case It looked at the mxf file Extracted those metadata stuff and then he sends back the metadata and the object data stream Which is basically exactly what was uploaded if we're thinking about the demo, right? We just extracted additional metadata and then We just complete the proxy put flow with no other changes So this data is gonna be replicated and all the good all the things that had needs to happen on a put path in Swift Right, so let's move to the other use case or the other way to invoke stall it's so In this case we've all the stall it upon get So in the in the demo this was where we actually Calculated the loudness feature So we've done it upon get whenever we got the we asked to do get plus this X run stall it loudest calculator To stall it what actually we got back was this loudness value, but in the general case this would be when you want to You're not interested in the actual data be saved on the on the Swift Storage system, but rather you want the transformation over it. So again Examples might be again decompression compression de-encryption whatever An example that we've been playing a lot is anonymization Think about a use case where you are where you have Medical records inside those objects and you want to de-identify them before they're being uploaded for a research or something So this is again something that can be done with a stall it upon get it's like a classical case Right, so let's see what happens In the get scenario so the request hits the proxy server The proxy server as it does with any get request is looking up a storage server Where where there is a replica of the object once you find such a? Such a server the request is being routed to that server again nothing new here so far Let's click and now we're going to zoom in into the storage nodes right, so once again the Object server middleware on the left-hand side the docker container on the right-hand side intercept the request For those of you who have more deep familiarity with swift we basically intercept the request After the object server application ran, so basically we're getting the content of the object We intercept the request when it's on its way back, so we already have the data there We over already have a stream that with the data of the object. That's like side comment So once again similar to the put we invoke the demon if needed We passed the the fd's The stolid sends back the object metadata and data in this case by the way all we passed back was the loudness Number and we continue with the get flow as if it was just a regular get Okay, so let's move on to the third way to invoke stolids which is on a post So in the get scenario and in the put scenario we were actually acting on the object that appeared in the uri Right in the put scenario we went and created an object according to what was there in the uri in the get scenario We were looking at the object that was in the uri Here we can actually invoke We can actually act on on one object or more that are already inside the system We can run a stoll it over them and that stoll it actually can can create other objects as a result of that so a typical body would look like execute the loudness to realization stoll it over the 15 seconds I mixf which resides in the demo project container and place the output inside the demo project container and In fact we can in that body we can list more than one object In which case there will be a stoll it instance running pair objects that is listed there And we can even do this for different stoll it's Right, so what would happen here is that the proxy? One one one on back. Thank you so The proxy would gather requests would parse it would understand how many invocation of stoll it it needs to it needs to be done and In this case there are three invocations. It's not it's like an Optionally there can be more than one as I've mentioned and it forwards three requests to three different object servers Okay, so don't don't get confused. There is only one One one object that I've mentioned in the body here, but the errors shows as if I've actually placed there three Okay so Next slide please So I guess a lot of people in the audience are asking okay. What about zero VM? So I'm sure many of you are aware of the OVM Which is by the way a very cool project which I really like and So what's the difference so I'll talk about two aspects here one aspect of course is the underlying sandboxing technology, so The zero VM guys are using the Google knuckle based zero VM Which basically means that anything that you want to run inside that sandbox need to be compiled in a specific tool chain that Probably gives you more security On the other hand using Docker containers there is much more flexibility in the ability to use off-the-shelf Software stack you can put it on the image of the Docker container and then write something in Java or in Python Which I think makes life a lot more easier for developers so I'll be out of my league if I'll try to compare like the security features of features of both technologies So I'll wrap I'll end here my first comment regarding zero VM The next comment I have about zero VM is that they've they've really I think they've done it a really nice Engineering work in actually turning Swift into a compute platform. They can actually They can actually make requests that are being translated to a whole workflow between Install it between in their case it's zero VMs that can communicate and sort of doing a pipeline They can do map reduce jobs with that. It's really nice And although this is something that we can do I mean there is no like theoretical or any reason that I can think of why we can't do that here We do believe that that we probably need a slightly different model I'm not completely sure that we want such a flow to actually run inside a Swift cluster This this also changes a lot in the runtime behavior of Swift Studying lots of threads. They need to communicate etc. etc It might be that this orchestration actually needs to be done outside of Swift in a way that also by that allows To use maybe other compute resources or being able to run Analytic work, which is which would take data not only from Swift, but perhaps also from I don't know how do traditional databases whatever So this is what I had to what I had to say about store it's in zero VM Perhaps we'll jump to the Next slide about future plans So we want to make this Available for experimentation If anyone is interested or want more more information is welcome to contact either Alberto or myself These are our emails Surely we want to increase the number of toilets introducing more processing step to produce metadata and build an advanced search Application on them continue to enrich the metadata prototype with additional storage base workflows. So Alberto believed that we do need workflows there. So I might be wrong here Okay, so before before I'll jump to Take questions. I want to thank the technical staff here was very helpful with setting up the sky And then setting this all up. So thank you very much for the technical team and I'll be very happy if you have questions Right. So the matter the question was how we did the metadata search implementation. So It's it's it's two pieces of middleware one of them would Would take the swift metadata as it flows in and and send it to To what's what's what's the name of the database? I'd remember to some kind of database and then there is another middleware that would intercept search requests and Go to that database and fetch the answer. Okay. There was yep John correct, I have I Know I have no I have it I have it per account We're not sure that this is the right decision But on the other hand, we have no like real life experience. So we're starting with that So we're gonna have a docker container running for each account and each account is gonna have their Unix pipe socket through which we can do all the magic That's is correct.  So so if you were concerned about the Storage, so so they're gonna use there. So this is kind of on the upside More questions. Yes Yes, actually, it's an array of input stream and output stream, but I kind of simplified it here So what do you mean by you? Oh, you mean by random random access? Right, so in some okay, so in some cases you can do that in other cases you cannot do that With the current prototype you can do this at all But but potentially with the input stream of the object You can probably do that because under the hood actually there is a file descriptor file there Right the file that actually holds within Swift the object, which is Run can be random access But of course the output cannot be because this is going to be streamed into the response That's gonna go back to the user on the socket right, so So I should have said that this is of course tailored for stream processing, right? if you want to Merge sort the file that's that's not the right tool to do that, right so In fact, so this is kind of an open question between ourselves We haven't blocked the ability to create local files Right, but then for me it seems to defeat the purpose I mean if you're gonna write a local file and then stream it back there So you basically done nothing you can at the same I mean it would probably take the same latency to just download the file and do this on the client side You kind of missed the whole purpose Anyway, this is the way I see it. Okay, so probably need to give this more thought. Yes Yes, and Unfortunately to be honest you probably won't get a very fast answer saying okay. Go ahead tomorrow You can start playing with it. It would take us some time. We're having some internal difficulties We don't want to get into that now, so The open source question, of course. Yeah, I'm ready for that one so So let us separate let us first separate between the question of open source and the question of making this available for experimentation It's not the same thing, right? Okay, so the open source question again a very delicate issue It's being in internal Internal conversation within IBM. I cannot unfortunately. I cannot say anything and I don't have any any good news at this point And I cannot elaborate of course, and this is a time where I switch to my own personal view Or I say that personally I would love to open source it I would love to to build a community around it I would love to collaborate with zero VM on on converged APIs, but again, this is out of my hands Hopefully before that will have a positive answer, but that again. I really do not know Yes Can you talk about erasure coded on how where the compute would happen? right, so In fact Yeah, we've discussed this just an hour ago with the erasure code team here, so I think that okay, so I Think that there is more than one answer here So the the short answer would be okay We're gonna switch to do all the processing on the proxy where the where the Where the erasure coding code has reconstructed for us the file and we pay for an extra hop here On the internal network. That's the easy answer. I guess the more complex answer is that if you're using a systematic code and You somehow know where the data really resides you can and you can distribute your solids to probably work Independently on each chunk then you can probably do something another observation is Maybe you can start with the story nevermind, okay, let's These are the two answers I have right now. I actually I brought this question in in the zero VM design summit last time and The answer was let's not worry about it now But then it's gonna be real soon. So perhaps we should start worry about it Yeah, right. Yeah. Yeah, so so the same answer goes here, right? So we can either move it to the proxy code or if you're smart enough to know How your chunks looks like you can get from the assembly object where they are and actually Go and run the toilets on each of the chunks and then probably combine the answers I'm afraid that The little I've said about the toilets. I can't even say about that. So so the the So I wasn't part of the team that actually developed this this and so I really don't want to Say anything. Okay, but then Right, so so Albert did you get the question? Well, I heard the question, but I'm I have my voice turning back. Maybe it's Mike Okay, so I heard the terms of the question, but I'm not really sure to have understood that So I guess I'll try to to refrain that or okay So I guess the question is why go and complicate Swift adding all these functionality where you can probably Run this Alongside of Swift Perhaps in inside data center or even on the client side or you were referring to inside open stack Before the okay, so to act basically why not have a layer on top of Swift that does that which yeah God that's got that so there is a I think there is a good reason for doing this because metadata enrichment Which is a category of processing steps loudness normalization is one, but there are many others Is not something that starts and ends They say when you have the file and then you forget it is something that really Come seen again and again during that content life cycle So actually having say stored These bulky files in an object store like Swift and from time to time on demand Extract new kinds of metadata is something which is really Day-by-day in media production. So actually you don't have a one-shot metadata structure and forget and store but is very Lively and continuous operation of metadata structure. So it is really important to have the stuff stored there ideally forever and just make the code run and Extract new metadata as they are available as the new algorithms Rise as maybe you you can you can just re-run the same method destruction with a new tool because of as a better quality Or as a better performance. And so this is something that really happens Day-by-day, so content is not really dead once it is elaborated But it continues to be Processed access and enriches So maybe I can edit on top of that. So we had earlier a talk about Swift and and I Need help what was it? Swift and spark. Thank you So the point there was that you can basically filter a lot of the data as a spark actually, okay, I got it Okay Thank you. Okay. Are we out of time any more questions? Let me I think if there are more questions. I think we can all right So, thank you and thanks again to the technical team Alberto, thank you very much. You're welcome. Bye. Bye. Bye. Bye. Bye. Y'all