Welcome, everyone. In the previous example of the machine repair problem, we have seen how the when we apply the dynamic programming equation or the Bellman equation to find the optimal policy and the value function, we have to actually do a number of calculations. If you recall, we had just a two-step problem in which we had only two actions and two possible states and yet even in that sort of a problem, we had to just to find the value function at time step 1, which is the second time step, we had to go through eight different cases. If you recall here, this is case 1, then there is case 2, case 3, 4, 5, 6, 7, 8 and each of these cases involved an optimization in itself. So, because in this case, we had only two actions, the optimization was trivial, we were just comparing the cost of the two actions. But then in general, if once one has a larger number of alternatives to choose from or larger number of control actions, this optimization can also be very tedious. In other words, the entire problem ends up having a huge computational cost, and that cost comes about because the number of possible histories that one has to keep track of, which is essentially the, which is encapsulated in the information that you have at any point of time, that keeps growing with time. Now, in this case, we have a three-dimensional information vector and that if you have a longer history, this would be even larger and that would even that would therefore result in that many more possibilities for the value of the information vector and that many more computations that we need to do. So, therefore, a central question in problems of with imperfect state information is how do we reduce this particular complexity? In particular, is there some way by which we can keep track of much less information than what is present in the information vector? Is there something that we can keep track of which is more parsimonious for which does not grow in complexity as much as this particular quantity does? And then as a function of that, we might be able to reduce, find the optimal policy as a function of that particular quantity and that will then result in a reduction in complexity. This particular quantity is known by the term sufficient statistic.  So, the information vector hides within it a much more compact or compressed representation which comprises of the essential information that is needed for your particular task. The sufficient statistic is a concept form statistics, but it is used in control theory in order to mean the bare minimum information that you need to keep track of in order to do control in order to apply the optimal control action. We will soon see in the forthcoming lectures how the sufficient statistic plays an important role in all of these problems and what really is the sufficient statistic for various classes of problems. So, what we will do now is actually look at a particular another special case. This special case is that is the problem of linear systems with quadratic cause, but with imperfect state information. And here we will see that there is something much simpler that we can keep track of then what we have been doing so far and we will find that in some sense we would we would have discovered in some sense the sufficient statistic for this particular problem. So, the problem that we have is a comprises of a linear linear system. So, you have x k plus 1 which evolves as a k x k plus b k u k plus w k where k now belongs goes from 0 to n minus 1 and we have a quadratic cost which we want to minimize. So, we want to minimize the expectation of x n transpose q n x n plus k going from 0 to n minus 1 x k transpose q k x k plus u k transpose r k u k. Now, you would recollect that this this problem looks a lot similar to the one that we had studied before where we had perfect state information. In fact, there also we had a linear system with this in this kind of form and there also we had a quadratic cost of this form. I have deliberately not written out what the minimization here is over because I have not yet specified what the information is. So, the way this problem differs from the earlier problem with where we had perfect state information is because there is an element of partial or in imperfect state information here. So, at the beginning of each period k at the beginning of each period k we get an observation. The observation comes to us as in the following form it is the observation is given by z k equals c k x k plus v k here c k is a fixed known matrix. The terms here of all these terms here w k and v k these are noise c k are these are known and deterministic matrices. In general, this c k may not be invertible or even if it is then you would still not be able to use that fact in order to recover x k. So, consequently we do not know what the state is exactly and all we know is this observation z k. The similarly the matrices a k, b k these are also known and deterministic. We again assume that the q k matrices these are positive semi definite and we assume that r k are matrices are positive definite. Now, we since I have said that v k and w k are noise we will make further assumptions about this noise we will assume that these are independent. So, v w k v k are independent and 0 mean. So, they have mean 0 and we will assume that they have finite variance as well. So, as before we will assume that their variance is finite and we will assume that they are there is also an additional source of randomness which comes from the initial state. So, they are independent this is also they are also independent of the initial state which is denoted x 0. So, what we are doing therefore is again we are minimizing this cost now I can write out what we plan to do we want to minimize this cost over mu 0 to mu n minus 1 where we are choosing mu u k as a function mu k of I k and as before I k comprises of all the observations that you have up until that time and the control actions that you have taken up until the previous time. So, this completes our problem statement what we will now do is apply the dynamic programming equation to this particular problem. So, the dynamic programming we will apply the dynamic programming algorithm. So, the dynamic programming algorithm as before begins at time n minus 1 and we will write J n minus 1 of I n minus 1. Now, remember this is equal to the minimum over u n minus 1 the action that we plan to take of the following expression. It is a minimum of the sum of the expected the expected stage wise cost and the expected terminal cost. So, the expected stage wise the expectation of the stage wise cost is the stage wise cost at time n minus 1 is x n minus 1 transpose q n minus 1 x n minus 1 plus u n minus 1 transpose R n minus 1 u n minus 1 plus the terminal cost. But in place of the terminal remember the terminal cost here is x n transpose q n x n and what we will be doing is substituting for x n in terms of this equation. So, we will substitute for x n as in terms using this equation.  I can also write here the control action just to be explicit or I can we can proceed assuming that the control action is a parameter it does not matter the answer would be the same. The expectation here is a conditional expectation. So, the expected it is conditioned on i n minus 1 and you are taking expectation here over all the other things that are random remember then this is an imperfect information problem. So, x n minus 1 is now random unlike in the problem with perfect information. So, x n minus 1 is random w n minus 1 is also random all the information that we have at time n minus 1 is is encompassed in in i n minus 1. So, the realizations of of the observation noise are all present are somehow encapsulated in i n minus 1.  Now, w n minus 1 remember w n minus 1 these are independent they are independent and 0 mean. So, which means that given i n minus 1 the w k's and v k's are independent and 0 mean. So, given the so we already have that the if I look at the conditional expectation of w n minus 1 given i n minus 1 this quantity is simply is is actually just the in fact the mean of w n minus 1 itself and hence equal to equal to 0. So, this quantity is actually 0. So, now observing this let us make a few of let us note a few things. So, when we open up this particular quadratic quadratic here we would as before have terms that are linear in w n minus 1 we would have terms that are quadratic in w n minus 1 and have terms that are that do not have w n minus 1 at all. So, the the terms that are linear in w n minus 1 would the terms that are linear in w n minus 1 would be of the form where w n minus 1 is being multiplied by q and then multiplied by a n minus 1 x n minus 1 plus b n minus 1 u n minus 1.  Now this is the sorry I should note this condition on i n minus 1 is equal to 0 and the reason for this is as before remember that x n minus 1 here x n minus 1 is driven by the control actions and the noise that has happened up until time n minus 1 it is therefore therefore it is cannot be affected by the noise that happens at time n minus 1 itself. So, this remember in our model the the the the noise this noise occurs after the realization of x k. So, it does not actually it is independent it is it is it is not affected by anything that has happened up until then. So, and because it is independent of of all the previous sources of noise and the previous and the previous observation noise as well this this w n minus 1 here is independent of all of this. So, as a consequence of this the this expectation here is simply the will you can see is will be the expectation of this given i n minus 1 and the rest of it given i n minus 1. So, long story short this this entire expression actually evaluates to 0. So, as a result of this I can simplify the simplify what I have written about and I can get this smaller expression for j n minus 1 as a function of i n minus 1 that expression has a minimization over u n minus 1 then you have you have the expectation of x n minus 1 transpose q n minus 1 x n minus 1 plus u n minus 1 transpose r n minus 1 u n minus 1 plus now the terms that remain are remember we have we would have a quadratic in w n minus 1. So, let us write that out first. So, this is you have expectation of w n minus 1 transpose q n w n minus 1 this kind of term we have seen before it had come up even in the problem with perfect state information. So, I have dropped the conditioning on i n minus 1 because the w n minus 1 is in fact independent of everything that has happened in the past. So, so as a consequence I can drop the conditional the conditioned conditioning on i n minus 1 then what I am left with is only this term and then I have a third term the third term is the third term involves now what we can do is the following we can we can pull out terms here that that involve u n minus 1 and do not involve u n minus 1. So, when we multiply when we multiply this particular the this term here with this term we can separate out terms that depend on u n minus 1 and those that do not. So, the ones that do not let me write those first. So, I have x a n minus 1 x n minus 1 the whole transpose q n a n minus 1 x n minus 1 and then I have my term which depends on u n u n minus 1 as well. So, that would be u n. So, that term is u n minus 1 transpose or maybe let me write it in a in the following way.  So, now here so what I have written out in so far is a is a is a product of these this term with this term. So, what we now need to write is the product of of this the product of this these highlighted terms with this underlined term. So, let us write write that out.  And remember since all of this is inside the expectation any I have my conditioning on i n minus 1 outside and then I close the bracket and then this is being minimized over u n minus 1 all right. So, there is some the reason we have written this out in the following form is that it affords some possibility for consolidation of some terms. So, for instance we can we can observe that this is a quadratic in x n minus 1 this is also a quadratic in x n minus 1. So, these two can be merged and written together we can also observe that we have we have a a quadratic in u n minus 1. So, that would be so we have a quadratic in u n minus 1 here and a quadratic in u n minus 1 here. So, these two can also be merged together. Now the there is in addition to this a linear term in u n minus 1 now the linear term in u n minus 1 is the one that need that we need to pay a little bit of careful attention to that is this term. So, this term is it has u n minus 1 appearing linearly and then multiplied by x n minus 1 which also appears linearly. So, let us take these terms one by one and start putting them together and then we will we will then come back to this. So, what we have here so let us let us put the quadratics in x n minus 1 first together. So, we have the so we have this particular you are minimizing this with respect to u n minus 1 we have the following expectation of the quadratics x n minus 1. So, x n minus 1 transpose I am just skipping a few steps here a n minus 1 transpose q n q n a n minus 1 plus q n minus 1 x n minus 1. So, these have been consolidated together remember the this expectation is conditioned again on i n minus 1. So, then I have my quadratic in w which so this is w n minus 1 transpose q n w n minus 1 expectation of that and then I have my terms in u. Now, let us go first we can combine the quadratics in u. So, let us combine those two.  Now, we are left with finally we have dealt with the yellow terms the blue terms and this un-underlined term. Now, let us look at the orange term. So, the orange term here has is linear in u n minus 1 and linear in x n minus 1. So, remember all of this is still inside this expectation out here. So, remember u n minus 1 was taken as a parameter. So, u n minus 1 is taken as a parameter. So, therefore it comes out of the expectation and then therefore so what we are left with inside the expectation then is just q n minus 1 a n minus 1 are constant. So, what we are left behind with what is inside the expectation is the expectation of x n minus 1 given i n minus 1. So, we have this term plus I think I have forgotten twice here. So, let me correct this here. So, when I when we take the cross terms we also get we get 2 here. So, we get this plus twice let me write it in the following way we have conditional expectation of x n minus 1 given i n minus 1 the whole transpose a n minus 1 transpose q n b n minus 1 u n minus 1. So, now the notice that notice that the middle term here has own is dependent only on u n minus 1 and since u n minus 1 was a parameter for the expectation this is actually comes out of the expectation. So, there is no need to put an expectation around this particular term. The expectation in the in the final term has eventually collapsed to an expectation only over x n minus 1. So, that is what has given us this expression here. The remaining here is now also outside the expectation because it is independent it is only a parameter. So, what we are left with is therefore, just a minimization of this complete expression with respect to u n minus 1. Now, this also we can simplify further by observing that this underlined orange term and this term they are in fact they do not depend on u n minus 1 at all. So, really the minimization over u n minus 1 is in fact instead of it being here it is actually the minimization can be put here. We can really put the minimization with respect to u n minus 1 out here. So, eventually what we are getting here is notice the following we are getting a quadratic in u n minus 1 who with a linear term that depends on the conditional expectation of the state at time n minus 1. Now, what we will do in the next part is solve for this and let us make some more observations on top of this.


back to another counter side episode today's episode we're going to be featuring Janus's strategy and build but before that if you wouldn't update it counter side videos please hit the subscribe button down below okay guys we are here right now featuring Janus future at war so Janus is actually a breath of fresh air since we've had a lot of awakened units so far counters and a little soldier in mech new releases recently so he is one of I think it's going to you know give us a new perspective on on PvE and on PvP as well so Janus is actually an SSR so he is a Ranger air type and he also dishes out an all-around attack okay so let's take a look at the stats for Janus as you can see here Janus has strengths a lot of strengths but don't be fooled because once he transforms into a defender he actually reverses some of those stats okay so right now as a you know as regular stats his strengths would be his HP which is above average at 45 4 4 2 attack is also a strength at 5 7 4 5 his defense is also high which is his strength as well at 21 26 and his crit is going to be one of his strength or last Trent which is above average his weakness actually is going to be evasion which is going to be obvious for mechs and hit rate is also below average which is not that bad can be you know can be mediated by gears so let's take a look at Janus's skills guys okay so when I told you that to let's say don't you know make a fuzz about his strengths because his strengths actually some of them drop down in whichever form he is in so let's take a look at his skill info and his basic skill which is his basic attack it's an AoE which has valid hits of 2 fires an energy cannon inflicting AoE damage on and around its target after transformation which transforms into a defender swings its enemy energy arms instead of inflicting AoE damage in front okay so basic attack maximum of a plus 20 up to level 5 okay passive actually gets interesting so let's talk about his first passive which is combat conversion system to activates its transformation system in case of emergency when deployed its defense is decreased by 90% so as you can see I told you guys even if his stat was was way up once is deployed he is still a ranger so his defense would naturally be below so it's it's going to only be 10% of what you saw or what is your existing main stat so when HP when it's HP drops to 50% or lower however transforms into a ground you ground defender with full defense he you know he recovers 90% but 50% lower attack and crit so again there's a lot of things you have to play out in terms of using Janus as his passive level 234 crit will be increased up to plus 20% and level 5 after transformation attack and crit is only a minus 30 instead of minus 50 so this is his first passive so again gears later will discuss on the appropriate gears in which which is for me is I think his most what is most suitable suitable for him whether it be on attack mode or in defense mode okay next would be his special skill which is Caliders arms it's an 18 second cooldown with valid hits of three fires a high powered energy beam inflicting a way damage on on and around its target after transformation choose an energy sword inflicting a way damage in front and forcing the enemies to attack it for five seconds which is actually not bad so again I'm actually tempted to discuss gears now but let's let's talk about it later so level 234 damage is going to be 25% additional for this skill and cooldown is a minus four seconds which will bring the cooldown to a good 14 seconds okay so the last one which is passive is janitor mode booster OS upon transformation cast a barrier equal to 50% of max HP and becomes immune to hit stun and special special or lower skills for this buyers duration so levels 234 this is going to be a plus 20% and for level 5 for buyer duration damage taken is going to be a minus 30% from Rangers and immunity to hit stun so he's going to be a good counter for Rangers once he hits defender mode but again we'll talk about his gear right now is to see if what are the most suitable gear for him okay guys as you can see here I was able to pull for Janus never mind the video I'll go straight to what happened I did pity Janus so that's it so I was not actually in the mood of recording polls so I did pull but eventually had to pity him so again I think it's okay because I had to burn my luck here in you know in one of the polls eventually because I've had good luck before and so far so this is actually one of my most bad polls to pity a SSR so as you can see guys in terms of his gear there are actually two types of gear sets that I would recommend whether you go with first we go with attack so if you go with attack I would recommend you put on a CDR gear set so that you could you could spam his special skill because again the cooldown for that is actually low but you could actually lower it and spam that skill and once he turns into a defender he would actually do more taunts because of that because of the special skill so I'm putting here CDR gear if you want to have him in a more tank here mode definitely go with HP because HP is going to be is going to have a big plus for her for his ultimate so those are the two gear sets for stats obviously go with CDR go with HP or actually go with hit so those are because hit is going to be below average it needs a little bit of a boost so might as well if you have you know if you consider if you want to consider the yellow part of the stats so let's open one so this part you want to change that again either CDR skill haste HP or you can change that to hit because again he needs hit stats so as you can see here I have hit in this gear I'm sure you guys have plenty of cooldown sets for mechs I'm sure you've saved up the radius so that is why I have completed his cooldown set because I have a lot of again very good very good cooldown sets this one has hit as well so might as well put hit on him because he needs that below average stat for hit to be up but for tier six gears if you have tier six might as well go again with the maize set or gorgeous crown set so anti ground damage resistance for maize it's usually for his weapon and for the accessories gorgeous crown is going to be more focused on hit which actually he needs so those are my tier six recommendations and it's a good thing that he was actually out because mechs really need a lot of reinforcement because again the meta is in PVP special is actually going crazy with a lot of awakens running around and let's go to his deployment guys okay so we are here in his deployment strategy preferably you deploy Janus right after you've deployed tanks in front so that you know he could at least survive more as a rager form so let's try to try to deploy Janus here try to summon ground and air units so again he really fires just a small burst so as for his special so that's it it's like a small photon torpedo which he fires he fires actually I think twice one oh no it's only once so again spammy skill he has at he actually has a slow attack speed it's better that that you actually take advantage of his special skill through CDR and I hope I could lower his HP so that he could transform see at this point I'm not sure if we can lower his HP and let him transform but again he's a good support support to start as a ranger then eventually mid game if he falls down he is a defender which has which has provoke or taunt which you know makes your enemies attack him so he's actually an overall utility a good addition to your mech team okay guys so my final thoughts for Janus so again a fresh you know addition to mechs well really well well timed addition because again the mechs need really a lot of love and reinforcement for PvP this guy is going to be insane we're paired with coffin 6 so we still have to you know wait for coffin 6 coffin 6 by the way is a ship and more or less for PvE once also shadow palace arrives this guy will be good as well but for now if you're if you're bringing mech teams and you need a reinforcement this is actually a must someone for you okay guys so so far we have discussed a lot with Janus I for those who have not yet pulled for Janus hope you have more luck in your pulls than I did and for those who have stuck this far this video hopefully that you would consider subscribing and also click that bell icon you don't miss any countersight videos thank you guys stay safe take care this is the warden and I'm out of here


Hello, my name is Hannah Sjeld. It was almost right. Thank you. He rehearsed before. I am working on a crowdsourced project called Granny's Dancing on the Table. And my journey into crowdsourcing was through the distribution of my first feature, Nasty Old People, which we released with the Creative Commons license on the front page of the Pirate Bay. And it was there for three days on the front page, and we got over 50,000 downloads during those days. And people from all over the world engaged and contributed in the distribution in many different ways. I started the production of the movie with a bank loan of 10,000 euros. And people donated money through a simple PayPal account to help me pay the bank loan back. And this was a great story, but it happened so much more, which I didn't expect when we first released the movie. The first thing that happened is that they started to translate the subtitles into 17 different languages. The first subtitles we got was Portuguese, and it came in after only two days. People also started to remix the movie. For me, when we decided for Creative Commons, this was the hardest part for me, or the hard part, the only hard part, because the editing took so long, and when it was finished, I felt that I don't want anyone to change this, really. I wanted it to be as it ended up, and what will happen? How will they do? I didn't feel really safe with it at all, but when I got the first remix, I was really, really, truly happy to see that someone put so much time and effort into this story, this project that they actually wanted to do remix. So now I'm totally convinced, and I want to definitely keep working with remixing. What people also did was that they started to organize live screenings, and the first live screening happened in Vladivostok, which is kind of as far away from Sweden as you can get, and it took only one week for it to get to Vladivostok. And this is a picture from the screening in Kiev, where people also printed their own posters of the movie, and in Kiev, they called Nasty Old People for Swedish love. And now we come to Granny's Dancing on the Table, which is a transmedia project. We are five companies to work together, and it will end up in a movie game and live events. And when we started this process, I felt that I wanted to keep the relationship with all those people who had contributed to the distribution of Nasty Old People, but I also wanted to find out what would happen if we open up the process earlier and invite people to take part in the creative process, and not only in the distribution. So, first of all, we made a page on Facebook where we invited people to take part in the script writing process. I didn't release the script there, but I started to ask questions about the characters and the dialogue and the scenes, and I wanted to know if anyone had had the same experiences as my characters. And what really surprised me is that many people were really personal. They didn't just come with ideas and thoughts. They told their own stories of their own life and explored their own fears and their own feelings connected to the themes of the movie. And then I was thinking that, okay, sharing personal stories is really a great way to engage people really deeply into the story world, but also to create a strong community, because when you share personal stories, you also get to know each other, and the people inside the community get to know each other. And the next step was to create workshops, small workshops. It was the game designers who worked with the script-booking technique, so they used pictures for people to associate between those pictures and the themes of the movie and their own stories. And once again, I was surprised that people could come together for three hours and they didn't know each other from before, and get deeply into their own lives and also into the story world. And this has been really important for us during the development of the game. Right now, we are in the middle of our first live event. It's called Granny Day, and we encourage people to upload a picture of their Granny and to write a short story about her. And we have received stories from all around the world. One of the themes for Granny's Dancing on the Table is the search for identity, and to look into your past, to see what happened in the past and how has that affected me in my life now. And it's about this girl, Ayni, who goes on a journey and searches for her Granny. And we wanted people not only to take part in to create the story about Ayni and contribute with ideas for this, we wanted them also to go on a journey with Ayni and to look into their own pasts and see who's my Granny and how did my Granny's story affect me? Because then you do the experience yourself that the story is about, and we hope that the people will get an even deeper experience within the story world while looking at their own lives. And I really love this one. I thought when I got this one, I thought, oh, I'm going to see that forest. And also, this one is very inspiring for me because I was confronted with my own prejudices. I didn't really expect to find that story behind this picture. This is a story about an amazing woman who came to Sweden. She wasn't allowed to go to school. She came from Iraq, but today she's a political fighter and she encourages her grandchildren to educate themselves and she's an inspiration for many people. So this way I think it's also like collecting stories that tells a female story that hasn't been told or isn't told so very often. We use those pictures in a street exhibition. We simply print the pictures and the text together and put them up. And right now it's going on in Malmö this week. It's the first exhibition that we have.  We have been contacted by people who wants to host their own exhibitions in their own hometowns, towns. So we just simply send them the pictures and they can do whatever they want with them. So the first country is Serbia now in two weeks. They will hold the exhibition and then he will send it, pass it on to the next country and so on. So why do I want to work with crowdsourcing? Proudsourcing in this way because it's a lot about sharing personal stories for other people. And I think that for me it's first of all because we need each other's stories. I believe in the power of storytelling and when we tell our stories and when we listen to other stories we also understand each other and ourselves much better. And we also, when we create the space where people can tell their stories we kind of facilitate meetings between people, true meetings. This also connects to the issue about who's telling what for whom. I mean now for this, for Granny's Dancing we have the contributors are mainly female 30 plus which is a group that is maybe not always considered so much as the contributors in this internet world. And I think it's great that we managed to reach out to this group but also that stories that maybe not is heard. Like many of those Granny stories that doesn't have a room always in the culture, they can come out and they can have a space when all people are invited to tell their stories within a story world. And of course it's, I mean basically just also a marketing tool because if people contribute to the story world of course they will want their friends and family to take part and participate in different ways to watch the movie and play the game and so on. And all this process has also made me think a lot about process and result and the contradiction in between because for me as a filmmaker I am often very used to look at the result. What's on the screen is what matters, nothing else. I mean the process is just what you're doing to get the result. At the same time I mean for me personally also the process is everything but when it comes to the relationship to the audience it's normally it's only the result that counts. But now during this work with crowdsourcing the process becomes much more important and the process is also part of the result because what's been written on Facebook the stories that people have shared or the granny stories and the stories for the script, they exist and they are important and important to me and to other people and to those who shared their stories and they will be there no matter the result of the movie and the game. And I mean of course I want to have a fantastic movie and a fantastic game but I mean I love the fact that the process can take so much place and be so much a part of the result because it's in the process that the relationships happens anyway I think. So I want to finish with just a few words about the price for crowdsourcing or how to say because I mean it's not for free it takes a lot of time and it takes a lot of effort to engage people to contribute, to find the page and to actually like for Granny Day actually scan in the picture and upload it on Facebook and write the stories. And sometimes I think to myself because it's so much to do I mean in the story world and we are also making the movie and everything and I think to myself is it worth it because it doesn't give any money back for example at least not right now in the beginning at this moment. But I think somehow that crowdsourcing could be part of our journey against sustainable development because if we look at the social and the culture values and consider them to be as high as the economical values then for me at least it's totally worth the time and the effort. My name is Hannah Sjeld and if you want to take a look at the project you can go to Granny's Dancing on Facebook or Granny'sDancing.com Thank you.