So, there is something known as the flow net in the earthen dam. So, you must have realized that having proven that the top flow line or the free attic line is a parabola and hence the equipotential lines are also going to be a parabola. So, the way we draw this is, if I consider that this is the dam, if I have the top free attic line like this, the next free attic line like this and like this, the equipotential lines to this would be perpendicular to the flow lines. So, this is how the flow net gets developed. So, this is the psi function and this is the phi function. What you are observing here is that this is going to be a set of confocal parabola. Now, suppose I consider this point, now this is what is going to be the bottom flow line and this if this is connected to the atmosphere, this is going to be the equipotential line. So, at point A, I have the singularity that means on the left hand side, I have the flow line on right hand side, I have the equipotential line and hence point A becomes singular. Now, this type of a situation can be created by putting a filter layer. Now the moment you put a filter layer, this becomes an equipotential, the atmospheric pressure prevails over here and hence the total head is going to be 0 at this point. The way we have plotted the flow lines and we have assumed this to be the bottom most flow line, all the equipotential lines, when you plot them in totality would extend up to this regime and this line would become the bottom most equipotential line, clear? So, if at point A, if I enlarge things, this is how it will look like. So, I have the flow lines coming and meeting over here and the equipotential lines starting from here, this is what the representation of the confocal parabola would be. You can also assume that this is impervious surface and this is the pervious, I will say this is the pervious domain, this is the impervious domain. Now, we will take the situation where the two problems which I wanted to discuss related to the confined flow, I hope you must have realized it is very difficult to draw the flow nets in case of the embankment by hand, one has to be a very good artist, otherwise extremely difficult, this is very complicated set of the flow nets. So, suppose there is a situation like you have an embankment, there is a filter layer, what I have done is a close look at this figure will show that I have shifted the point A inside the body of the dam and that answers the question which she is asking that how would you control the seepage lines so that they do not cut the downstream slope. So, truly speaking, this is the engineering solution which we are providing by shifting the entire layer of the drainage media inside the body of the dam. So, what is going to happen the moment I shift this filter layer somewhere here, point A becomes the focus of point A is the focus of the parabolae, the moment it goes over here all the flow lines will get diverted towards the filter, this is the engineering solution which is given to make dams stable against seepage alright and at the time of construction itself this can be done. So, imagine if you want to construct a dam, how would you do it? You will first lay in this portion certain layer of granular material, you will start laying the soil on the side and compacting it, so this is the soil, second layer of soil, second layer of gravel or sands and then you can give it a shape of the final embankment like this. So, this becomes my drainage layer and point A get shifted inside, now the moment I have done this type of a manipulation, the top flow lines will always come and cut at the vertex, so this is the vertex point E, now if you use the concepts of geometry it so happens that AE is always the mid of suppose if I take F point where AE is equal to EF and if I draw a line which is perpendicular to the surface what this line would be, this will be the directrix clear and you know the property of the directrix, so suppose if I define this as Z naught or whatever let us say if I define this thing as Z naught and if I consider a point here at y and x, so the moment I have fixed the directrix I know the focus, I know the vertex E and I am assuming that AE is equal to EF, total length of the filter is known y square plus x square under root will be equal to x plus Z naught, this is okay, this is the property of the directrix, any point sitting on the parabola would be having would has to satisfy this condition, is this okay, so what I have done now, I know this point, I know the starting point as 0.3 times L okay where this is L, this is the point of submergence for solving the parabola how many points we require, 3, one is known, second one is fixed, third one I can obtain by using this geometry, so if I have if I know the Z naught value, I can consider a point over here, this is the equation which I get fine, this discussion is mostly based on this construction, so if I know the Kx and Kz value I think this is the problem which we have already discussed in the class, homogeneous soil mass, anisotropic in the sense because hydraulic conductives are different in x and z direction, what you will have to do is you will have to go for equivalent hydraulic conductivity which is root of Kx, Kz, what is causing flow H, H is known as 18 meters, so this 18 meter is going to cause the flow, after the flow net is constructed I can count the NF and ND and this is 3.8 and 18, this function is transformation of x into x t direction, so root of Kz by Kx would be 0.6 times x, all the dimensions in the x direction will get reduced and hence you are shift seeing that the embankment has become steep, GCT is nothing but 0.3 times HCT, if height is 18, this is 45, 45 into 0.6 would be 27, apply the correction again 0.3, so this becomes 0.3 times 27, if the flow net is done I can draw a line passing through A in the vertical direction, wherever this cuts the top flow line, this becomes my A prime, A A prime distance I can measure on the filter bed, I can draw a line passing through this, this will become direct mix, okay, once you have done this construction first all the things are simple. So if I know the NF and ND, if I multiply by the H value I can find out the discharge taking place through this section, what he was asking I have tried to answer over here, at a given point P if I want to know what is the total power of pressure, the thumb rule is if the 2 equipotential lines are known, you interpolate in between, draw the equipotential which is passing through the point P, draw a horizontal layer, if I put a piezometer here it will go and cut the top surface at this point where the equipotential is cutting the top periodic line, point of intersection of the top periodic line and this equipotential is going to give you the total head at this point. So once you have got the value of u, you can compute whatever has been asked, this is okay, I will repeat it again, after drawing the flow net, if I want to find out power of pressure at any given point what I have to do is I have to interpolate the equipotential line passing through that point, find out where this equipotential cuts the top periodic line, the vertical distance between this point and the horizontal plane which is passing through point P is nothing but the power of pressure, this is a good example of unconfined flow under homogeneous conditions, anisotropic conditions and you can solve the problems, practice it by following some book. The last thing which I want to discuss regarding the unconfined flow is most of the time we come across a situation where we have different zones in the embankments. So suppose I want to reduce the you know seepage which is taking place through the body of the dam, so there are two ways, one I did is by maximizing the H value and getting the value of q which is a inverse process, another better way would be if I design let us say a core of a impervious material in between, so suppose this is the upstream water level and if I want to check the seepage taking place through this, what I have to do is I have to put a impervious core, now this becomes the situation which you have already come across in physics when you are dealing with the Snell's law in case of light, you know. If I consider this permeability as k1, this permeability as k2 and if you realize how the flow lines are going to travel through them, what is going to happen? This flow line comes hits the second layer of a different permeability, the same thing is valid when we go from k2 to k3, so the moment this comes out of the system depending upon k3 there is going to be a deviation in the top flow line, so this becomes a case of the interface. Another type of interface might occur if I consider a dam where I have provided a tow filter here. In this case, k2 is tending to 0 because I wanted to make system impervious, so I have used clay to make a core of the dam through which nothing should permeate, very small value of k2, but here I am sure you will realizing that k2 is tending to infinity because this is made up of rock cores, gravels, sands, there is a specific purpose, now this is the filter clear, what is going to happen? The top flow line will come hit across this and then it enters into k2 whether it will enter into k2 or not is the big question, so when the flow is taking place across the discontinuities or the boundaries, what you have to do is you have to solve this problem by assuming a simple model and let this model be like this that this is the interface where the two materials are located, this is let us say A material, B material, if I draw a channel of flow clear, so this is the material A, permeability is kA, this is material B, permeability is kB. Now the question is you have to find out what is going to happen to these flow lines when the interface is there, so draw the perpendicular to these interfaces and then if I ask you to draw the flow net it can be done, this angle is alpha A, angle of incidence of the flow line in the media A, this angle is alpha B, the angle of deviation with respect to perpendicular drawn on the interface, I will show interface a thick line, I can draw the flow nets, this is the first equipotential line, second equipotential line, third equipotential line, I can assume this to be let us say YA LA, so YA is the width of the channel, LA is the length of the channel, same thing can be done over here also, so this is YB and LB, can you draw a relationship between alpha A, alpha beta, derive a relationship between this XA, sorry YA LA, YB, LB and what is causing the flow to take place, delta H, so I can have a situation where delta HA and delta HB are provided, the continuity says that discharge will remain same, so if discharge remains same QA will be equal to QB that means this will be QKA multiplied by delta HA upon LA into YA into 1 and this will be equal to KB delta HB upon LB YB into 1, fine, continuity equation, try to solve this expression and hope you will realize that what you will be getting is tan alpha A upon tan alpha B equal to KA upon KB, of course you have to assume some relationship between YA and YB, LA and LB, so if I assume both sides the squares YA LA cancels out, YB, LB cancels out, this is one of the solutions, these are perfect disquires, these are perfect disquires, clear, so I have simplified the whole thing and I am getting this expression, now what is the interpretation of this expression, that is very important, so the whole story started from the interface problems like this, this K1 is a finite value, the value of hydraulic conductivity for the filter layer is tending to infinity, clear, so what is the relationship between alpha 1 and alpha 2, alpha 2 is going to be 0, why, what is the meaning of this, the seepage line will never enter in a media which is highly permeable and hence what is going to happen, the flow is going to take place parallel to the plane of the filter, it bypasses the media, now this concept is used for designing most of the filters in industry as well as in various civil engineering projects, so coming back to this situation what we have been talking about, I think I have given you the basics and you can extend them to this situation, K1, K2 permeability contrast, we have zone 1, zone 2, what I have done is zone 2, you have to go for equivalent permeability transformation, flow channels will remain same 1, 2, 3.6, 1, 2, 3.6, the same thing is going to enter into this also, number of drops will not get affected because of the hydraulic conductivity, I hope you agree with this because number of drops have nothing to do with the permeability of the material, they follow the geometry, so here you have the equipotential of 100% and here you have equipotential of 0% because this is a filter layer, so 16 head value is getting dissipated from this point to this point, how many head drops? 1, 2, 3, 4, 5, 6, 7, 8, clear, so I have got NF by ND, NF by ND is a constant term we discussed in the previous lecture and if I multiply it by the AA prime value, you remember AA prime value, this is nothing but the distance of the focus from the directrix, so this can be obtained very easily, so though these problems look very complicated, they are very easy to solve, the moment you have got NF upon ND, the moment you know the focus point which is intentionally done by you, clear, you know the top flow line, get the value of AA prime, directrix is known, do a little bit of computation by using that expression and hence the discharge across the plane can be obtained very easily, so NF upon ND multiplied by AA prime is the discharge through the body of the dam, clear, so in this case, it is a beautiful example of how two zones have been clubbed together to do the seepage analysis, so I am using the concepts of isotropic systems homogeneous individually, but I have two zones which are heterogeneous in nature and I am showing here the concept what we have used over here that there are two ways of solving this, either I should be having a relationship between delta HA and delta HB, suppose this is X, not X, suppose some alpha value, alright, so what I am doing is I am defining the relationship between the head drop across first layer and head drop across second layer and how they are related, I will get one solution, clear. The second way of doing this would be the geometry of the system, so I may say that YA upon LA is equal to YB upon LB multiplied by some beta, this is my choice because this is a geometry, I can control the geometry of the flow or I can assume the distribution of the heads across this point. Ideal situation would be if you keep the head constant between this and this, there is no problem, that is what actually we have done in this case. In the previous lecture, I had stopped after discussing the case of the sheet piles and I had emphasized on the concepts of determining the discharge through the opening below the sheet pile and the impervious strata. And this is where I had talked about how to compute the number of drops and number of flows which you can utilize to obtain the discharge. There is another class of problem which normally is discussed in the realm of seepage analysis and that is what is depicted over here. So, there is a sheet pile which is retaining a certain height of water 4.5 meter and there is a tail water or the downstream water and the datum has been considered over here with 0.5 and I am interested in finding out what is the pore or pressure at a given point in the porous media. So, once you have obtained the flow net, it should not be difficult for you to compute the pore water pressure at a given point because remember the entire analysis we have done until now is by assuming the delta H which is the total head and that causes a seepage regime to get developed into the porous media. So, delta H is H1-H2 that is 4 and this head is getting distributed in the porous media. So, at this point P if I know the equipotential, I know the total head. If the datum is known, I know the position head of point P which will be equal to minus Zp and if I insert a piezometric tube over here, the total height of the water column in this tube would be this much that is HP plus minus Zp alright. So, this is how we can compute the pore water pressure at a given point. Rest remains same and you can compute the discharge which is taking place through this orifice or the open space. I will move on to another interesting problem. There is a sheet pile and I would like to find out what is the pressure which is getting exerted on the sheet pile along its depth. So, this pressure is because of the seepage. In the second course when we are dealing with the earth pressure theory, we will find out the earth pressure which is coming on the sheet pile and then the combination of the two will give you the total pressure which is acting on the system. The concepts remain same. There is a sheet pile, this is the downstream ground surface, this is the upstream ground surface. There is a water table here in the backfill of the sheet pile. At steady state condition, the flow regime develops in the porous media and the height which is attained by the downstream water table is 4.0 above the ground surface. If the flow net is known, you can work out the number of drops at 0.1234567 along the sheet pile, I can work out the pore water pressure and just now I have discussed how to obtain the pore water pressure at a given point. Remember, each equipotential line defines the total head. So, if you know the elevation head of the point, total head minus elevation head will give you the pore water pressure. It is a simple example which I thought I will discuss over here. These problems I have taken from the Craig's book. So, my suggestion is please follow either Craig book or any other book whatever you might be having access of. Another interesting problem and this is something to do with what you were asking in the last lecture. There is a body of the dam or a veer in hydraulic scores you must have studied the design of veers. So, this is the cross section of the dam which is retaining water with a water head of 5 meter and downstream side you have a water tail of 1 meter. As the thumb rule is that tail water level is considered as that data, draw the flow net number 1, find out the NF ND. So, I think now you can realize that drop number 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15. So, I have drawn ND as 15. The flow lines would be 1, 2, 3, 4 and then some fraction of the squares which I am considering as 0.7. So, number of flow lines I have taken as 4.7. You can compute Q value as usual and this is a discharge just taking place through the bottom end of the sheet pile you may say on the opening. I can find out this discharge across any element. So, suppose if this is the element across which I want to find out the discharge what I should be knowing is what is the head difference across any element and then NF upon ND I will be using and then I can compute the Q value. What I want to highlight over here is these type of analysis are done to find out the uplift due to the seepage. If you remember when we are talking about the seepage force computation, I had drawn a figure where I had created a earthen dam and then at the base of the earthen dam I had shown you that this is a seepage pressure which is acting. Suppose you have to find out the pore water pressure at 0.123 and so on. Now, this is a pore water pressure which is acting at the base of the earthen dam or any system which is sitting on the soil media. 0.1, the elevation head is 0.8, all right. Datum is at 1, so the elevation head at 0.1 is minus 1.8. If the equipotential line which is passing through 0.1 is known, I know the total head at this point, I can compute the pore water pressure H minus elevation head. It might so happen. Suppose if I want to find out the pore water pressure between 3 and 4, let us say some 0.3 prime, what I will have to do? I will have to refine my flow net in such a manner that one of the equipotentials passes through 3 prime. So, in that case, the whole flow net will get changed. This is what somebody was discussing yesterday that and I was talking about the precision of the flow nets, all right. So, please practice these situations, follow some book and I think there should not be any problem. Once you know the pore water pressures, you can obtain the distribution of the pore water pressure which is acting at the base of the system and then I can impose this pressure for finding out the stability of the dam. Now with this, I finish my discussion on confined flows. As I said in most of the cases here, whatever we have discussed until now, the porous media is confined by the boundary conditions, all right. Now I am going to take the situations where unconfined flow takes place.


Meet me outside Chatrath, retrospect. Chat, do you guys remember Deputy? Deputy is leveling his honor. And he says he's getting camped by a level 70 blood off rat paladin. I'm- I'm like tank spec though, I don't know if I can kill him. Let's see. Uh, I just- I'll be there soon.  You're normally invincible. Eight of them ganked me. I killed four of them. Yeah, I accidentally hit one of my mouse buttons. I- I auto-dismounted. Okay, look at that. 498 d- I mean, oh, dude, I am a beefy. Dude, this is so cringe. This guy is literally- This paladin is camping a level 6200. Look at him. Look at him. Carnival slug. Dude, he's just hammer-wrapped. He's not even hammer-wrapping him. That is funny. Dude, what a try-hard. Oh, dude. I have no mana. Hold on.  Bitch. Oh, nice. Nice- yeah, nice freedom on your justice, buddy. Dude, you're ganking. You're camping a level 62. Dude, that was so funny. This is what I'm saying.  Look, it's just a little bit of Paladin of Police force, okay? Just a little bit of Paladin of Police action. That's all it is. I love how he's trying to hit me. I'm literally just dancing around there. Trying to freedom seal a justice. Oh, to be fair, I'll be fair about the freedom seal a justice thing. It's, like, that happens sometimes, right? Like, it's just your brain reaction. Hey, buddy. What's up, man? What are you up to? Wait, did he log out? I think he logged out. Because there's no skeleton. Either that he either logged out or layered. I will say this, though.  When you let low levels survive, and then they- they end up coming back and causing problems for you, right? Like, that druid came for that pal- I mean, I didn't die, but still. I'm getting camped. Somebody invite me for a layer. These alliance have nothing to do all day. Net Paladin leave?  As soon as somebody came back to gank him, he just- he just layered out and left. You know what's the funny part about this? We're not even- look, these guys are AFK. We don't even touch them. We're just like, we don't care. Please stop murdering me. Wait, which one are you? Why are you ganking low levels? Or are you the 63 Shaman? Kazrax, you're the 63 Shaman. Why are you helping level 70s who are ganking children? They're- they're killing children. They're killing babies. They're- they're child murderers. You're guilty by association. I don't know. I just got here. Uh-huh. Like- like the story, son. Here son, I'm gonna go ahead and have to cuff you while I ask you some questions for your safety in my own. Okay, I think, uh- I think we made our point.  Oh! Oh! Oh! Oh! Oh! Oh, the Big Bad F-Fan came and stopped the guy ganking the level 62. Oh, the Big Bad F-Fan, so I had to get six of my buddies together and not kill him. Bring a priest next time.   I'll tell you that much.   Look, I'm just being 100% honest. And every single person who watches, sees it, and they're like, yep, now I know.  And like, listen, best case scenario?  Worst case scenario? Whatever just happened happened. You literally had a personal cockroach group healing you. You had a level 70 paladin ganking levels, camping a level 6200 for half an hour. Are you kidding me? You're all bad. Every single one of you are terrible at the game. You're wasting your own time. You're not good at anything. If you were, you'd actually be going to play the game instead of trying to ruin somebody else's gameplay.  Literally unsub. And you know what you can do when you unsub? Hey, here's what you can do when you unsub. You can spend that 15 bucks, and you can take a third of it, and you can continue your sub from me. So welcome to my stream.  Not thanking subs. S-Fan, thank you for the gifted sub. Thank you, S-Fan. Here's my thing.  The weird, cringy, stream-snipey, whatever, I don't deal with it. You're trying to defend yourself from trying to ruin somebody else's gameplay experience? I get it. You're allowed to be bad at the game. You have the freedom to be bad at the game. Don't try and ruin somebody else's gameplay experience. PvP server?  That's not PvP.  If anybody says it's just PvP, you should be ashamed of yourself. That's the most embarrassing thing you could possibly say.  At least give me the 1v1. At least let me run the gauntlet and try and 1v1 each one of you. Because I'll tell you, I probably won't be able to 1v1 each one of you. Not in this spec. But at least, hey, at least give me that. Then 1v1 you slob, dude.  Meet me outside Shatrath retrospect. Come outside Shatrath. I don't even know.  Come out here. And bring your friends. Let them come watch. Meet me at the flagpole right here after school. If you beat me 1v1 in a fair fight, this and that, I don't give a damn, right? That's the game. What I do care about is when you go and you start trying to camp people and this and that. So come here. Come here on your paladin. Bring your blood out.  Was he stream sniping? Yeah, he was. Which one? Which one was he? Or is he, is he getting his world buffs? Tell him world buffs don't work in Burning Crusade. Like, listen, sometimes you win. Sometimes you lose. That's life. Where you at, dude? He's fixing his spec. Dude, come on. Listen, I just got done rating. Come on. I don't got time. Listen, I'm literally in a red. Come on. Or not red. I'm pro at PvE. Like, I already know I'm probably not going to win, right? Like, you don't got to go and respec and do the whole thing. Hold on. Twinlab. Do you want to duel in the meantime? Okay, I'm ready. Dude, Gesh, Zugas, Luck, Belar, Nakaaz, Makka, Ag. Dude, I can speak Orcish. Alright. Good duel, sir. Good duel, sir. Good duel. Very good duel, sir. Oh, I had fire resist aura. Well, dude, it's because the block was hot. Dude, I got it. I had to have fire resist aura on. What? Hey, where's retro? Hey, where's retrospect? He's not coming. He's not coming. I'm telling you, it's not that big of a deal. I'm PvE tank. I'm going to lose some of these, okay? It's just the nature of it. Oh, there he is. Okay, you ready? Alright, buddy. Here we go. Okay, a good clean fight. See, Twinlab is our ref. He wants a good clean fight, okay? Nothing crazy. None of this, none of that. It's all good, okay? Okay, when Twinlab leaves the center, that is when the duel begins. Okay? Yes, absolutely. I don't know what you're doing. I don't know how you're doing what you're doing, but A, one, two, three. Ding, ding, ding. Okay, very good. Okay. This is bad for me, actually. Oh, good duel, good duel, good duel, good duel. Very good, very good duel. Okay, you ready to go again? He had a 5% damage buff from the zone.  Like, to me, it's like, it's just, it's what it is, dude. When people start respect you to prove a point, you know they're mad? Yeah, well, it is what it is. I mean, like, I'm assuming he went in a respect, right? Like, that's why he went, that's why he's gone so long. He was killing me AOE pulling. Aren't you level 70 and he's level 62? My mage. Wait, the meme literally happened? The meme, you're telling me the meme literally happened. Well, guys, listen, so what did we learn today?  That's what we learned, okay?  Yeah, hunters and multishot, it's a huge issue. That's what we learned.


Hello everyone. Can everyone hear me okay? I sound British. That's just a thing James understands so it's good to have another Brit in the room you know for moral support. So I'm Chris. I love Elixir. I have been doing Elixir for a few years now, given a few different talks about some of the things I've done in the past. As Desmond already said, we host a podcast together. It's definitely worth checking out. We talk about a lot of the things that we've kind of been going over today as well. So might be worth listening to the back catalogue just you know and Desmond likes to plug his consulting business on there as well. So you can hear about that. And I also help organize the other MPEC. So the one that's in MYC, which hopefully we'll see some of you at and shout out to all the New York crew who are here today. So it's really good to see everyone. So I work at a company called Frame.io. I'm a director of engineering there where we basically build review and collaboration software for video teams. So we're used by all these big companies like Vias and Turner and stuff like that. And there's loads of stats here like 450,000 customers and we're founded in 2014 and we're based in MYC. And basically what I'm going to talk to you about today is quite a long journey that we've been on over the last year where we've gone from Ruby on Rails to Elixir and Phoenix. So I'm going to give you like a whirlwind tour of some of that journey. Like honestly, if they would have let me, I probably could have spoken for the whole day about some of the things we've been doing. And that's not because I like speaking. It's probably because there's just been a lot. So hopefully you'll get a little taste of that today and see some of the great use cases for Elixir in the wild as well. So let's start off by saying why did we do a rewrite and why was that in Elixir? So first of all, yes, we did that thing. I'm sure everyone's familiar with this blog post. Sorry, Joel. Yes, we did a rewrite. I'm going to try and justify some of the reasons why we did that today. And actually, Andrew mentioned this really, really well and succinctly earlier about, you know, software grows. Software grows over time and it gets complex and it gets messy. And when you're in a startup especially, like requirements change, right? You're working on one thing one week, the CEO comes to you the next week and he's like, you know what? We need to pivot. And often the code bases reflect some of that. And that's very, very much the position we're in at Frame.io with a lot of debt in the code base. So I'm just going to run you through some of the key issues that we had. So it was a Ruby 1.9.3 app running Rails 3.2. Two major versions out of date now for anyone who keeps up with the Rails community. So it had no tests, none at all for anything in the app whatsoever, which also coincidentally makes rewriting it quite difficult but I'll get to that in a bit. So it also had this custom RRM that was backed onto DynamoDB, which you might be thinking, well, that's quite unusual given that Rails comes with this framework called Active Record. And yes, it was quite unusual. So this talk's recorded. I'm going to feel really bad for our CTO. We wrote quite a lot of this but hopefully he won't watch. So there was a lot of metaprogramming everywhere across the app. It takes you a really long time to trace any calls to see the source of them. There was absolutely no logical separation of any aspects inside of the application. So being a collaboration app, it's all about what can you access and when and how. And there was none of that.  And also there's no metrics of visibility into the performance. Okay, so that's the API side. Let's just touch on the database side. So I mentioned that it backs into DynamoDB. So for those of you who are familiar with DynamoDB, it's a key value store, effectively like a NoSQL key value store. So it's really, really good for horizontally scaling except we weren't using it in that way whatsoever. So everything was a string, everything, even nulls. There's a value called JTNullValue which is a null apparently but also a string. All of our foreign keys were stored on the entity but encoded as a JSON list. So everything, nothing could be updated atomically. So we would actually drop customers data left, right and center if there was two updates happening at the same point in time. And DynamoDB costs us a lot of money to run this workload. So and then to top it all off, no pagination whatsoever because of the way DynamoDB was structured, which resulted in some of our largest customers having response times of up to 45 seconds. And that's no exaggeration. I literally had to up the load balance of time out to 45 seconds to make sure that we could deal with those requests. So yeah, that's kind of like it in a nutshell. I came into the company and we were all talking about doing this rewrite. And we really felt like we'd got to a point where moving forward wasn't a great option for us with this existing code base, right? And I'm sure many of you are in the room thinking, you know what, my code base is bad but I probably wouldn't rewrite it. And I completely understand that perspective as well. Rewrites are big, they're hairy, they're dangerous, they're costly, right? We decided that if we were going to make a dent into this thing, which is a monolithic API, that actually our best bet was actually starting over and rewriting it and carrying on. So why Elixir? Again, I think like the speakers here today have done a really fantastic job of arming all of you in the room who haven't used Elixir yet with a lot of the reasons why it's so fantastic. For us, we had a lot of Ruby and Python developers. I think the ramp up there is pretty straightforward. There's a lot of people who have made the transition. The concurrency was a key aspect of why we chose Elixir. And the fact that we're building on top of something that's been out there, as Emma so rightly pointed out earlier, just, you know, it's a fantastic thing that we're building on top of years of legacy and very well, very well architected system. And, you know, lovely language attributes as well. But the biggest reason was really that someone else had sold it and I was kind of just coming in and helping to bring that thing to life. But so next, I really want to like take you through what the system looked like and what we shipped and then give you a bit of a tour through some of the more kind of interesting parts of the system as well. So, sorry, this diagram might be quite small for those in the back hatch trying to fit in as many of the services as possible. So, really, we had this big monolithic API in the middle. It was a Rails API. Then we had a few node microservices around the edges that did a few different things inside of the app. But really, you can see this as one big monolithic system. There was a high degree of coupling between all of these different components. They all relied on the same schemas. They all relied on the same database, ultimately, going through DynamoDB, as I mentioned before. And to tell you the truth, some of these services in here as well were not great. Like the email digest service that's on the right hand side here, we had to restart every single morning.  So, the digest service in particular was the bane of my existence for quite a few months. Basically, woke up every single morning, go into EC2, click the restart button and hope for the best. But it kind of worked out. Yeah. So, we have all of these different components here. In terms of clients, we actually support, so we have an iOS app, we have a web application, we have these different embedded Adobe applications as well. So, there's a lot out there in the wild and things that we can't easily update as well. So, what we replaced in the system are all of these parts I highlighted here today. So, I mean, it was basically most of it and actually it was kind of all of it. And there was actually a bit more as well, but I won't go into that today because it's probably out of scope of this talk. But, you know, we really had to address this head on. So, we thought about it and we said, if we're going to tackle this, we're going to have to ship all of these different parts. And for us, what we actually shipped was this elixir powered API, a new notification system, a new real-time service, and this support tool so our customer support team could actually access all of this data. And then at the same time, what we did, it was we took all of our data from DynamoDB and we migrated that over to Postgres as well. So, we did this whole migration in about, it was like eight months from start to finish with a team of three developers, two of which had never worked in elixir before, if that's some kind of metric of adoption and how good it is. And as well as that, we dockerized the whole thing. So, like many of the other people on the deployment plan earlier, we run everything inside of Docker and we completely rebuilt our tooling from the ground up. So, this is a large project, something that I've spent a good amount of the last few months dealing with. And we actually shipped all of this in November. So, I can give you some real stats because it's actually been running in production.  So, I'm glad about that. So, what the system looks like after we shipped everything is a bit like this. So, again, this might be slightly difficult to see. But effectively in the middle here, what we have is a very large monorepo that contains everything inside of an umbrella application. We have a real-time service at the bottom that we actually shipped before we did any of this stuff. So, we swapped our socket IO, Node.js service with an Elixir Phoenix service so that we could understand what it takes to run Elixir and Phoenix in production. That was a really great learning experience for the team and be able to scale it and understand how we'd run it in Docker and all of these kind of ideas before we approached this big monolithic chunk. And then inside of the big monolith there, I don't know if you can read it at the back, but there's this part of the API on the left called Munger. And Munger for us is this kind of like intermediary API layer. And I'll explain a bit more about that later. But you can think of it like a consumer of the new API that acts as like a data translation layer between our clients and the new API itself. So, all of that bit in the middle was all Elixir. We separated out all of our core business logic. We moved everything to Postgres. We added a bit of Memcached in there as well. And yeah, this is kind of an overview of the architecture there. So, everyone wants to see some wins, right? So, I can reveal, the grand reveal, some really cool stats that happen to us. So, we were running around 40 EC2 instances before we made this migration. So, that was about 20 for our API service, about 20 for our Node.js real-time service. And what we did is we moved all of that down to five. So, we run everything now inside of five EC2 instances. They're about comparable in terms of the size. So, this is an okay comparison. The difference now is that we use Docker and ECS. So, we're packing all of those services into these instances, right? So, there's no more treating like our API service gets deployed over there and runs on its own server. We treat compute resource and memory as just one big allocation and you're putting services inside of it. So, that's how you get some of these wins as well. And if you haven't looked at stuff like ECS or Kubernetes or Mezos or any of those Docker kind of scheduling things, it's well worth a look. And I think it's like heretical to say, but I think the Erlang VM runs really well inside of there. I'm sure other people have different opinions. So, in terms of our response time, we're looking at about a 30 millisecond response time, about 120 requests per second. Pretty great. And that was down from about 300 milliseconds previously and going all the way up to the 95th percent I was actually a bit higher than that. Our database costs went down by 91%. That's largely because of the workload we were running on DynamoDB. And honestly, you can't attribute the DynamoDB part to Elixir. I know that. But it's a nice case study for using the right database for the job. And now we have about 70% test coverage across the board with about 2,000 tests. So, we feel safe in continuously deploying this thing, which is great. Before, we'd literally like cross our fingers and hope for the best and hope that every deploy worked out and we didn't bring down production. And visibility-wise, I'm sure if you're not running Elixir in production now, some of you might have questions around what do you do about like new relic substitutes and things like that. Well, for us, we actually just run statsD. We pipe everything through statsD to datadog. And then we get really great visibility through just looking at the memory utilization. And we use this tool called Recon as well, which allows you to get a really deep look into the scheduler and get some of those nice scheduler statistics out of it as well. And for me, some of the best things are that we started out this project with a clear goal about our architecture as well. So, we'd learned a lot of lessons from the previous API. And I think this goes without saying when you do a rewrite. You're kind of taking what was there before and thinking about how to improve it. And what we ended up with was a really clean modular documented, and I would say at this point, pretty maintainable code base.  Like, that authorization logic is now a separate module that we can test independently. All of our service entities are separate. All of our persistence layer and business logic are all separate. So, we can test those all in isolation. And for me, the best thing of all has been, we've been running this since November and literally touch wood, even though it's plastic. I haven't been woken up in the middle of the night and none of my team have as well. So, we're doing an insignificant amount of traffic through this service. It's not like web scale, but it's pretty high, right? Like, it's significant. And we have had no real large incidents yet. And it runs stably. The graphs look pretty consistent. And, yeah, just very few incidents in general so far. So, I wanted to give you a bit of a whirlwind tour through some of these parts of the system. So, we're going to go through these four parts here and hopefully they will be interesting to some of you who might be doing something like this yourself. So, first of all, the intermediary API. So, as I said before, it's kind of a consumer that consumes our new API and maintains the contract between the legacy API and our clients. And that contract means basically stringifying all the things, which isn't that fun, but someone had to do it. And what that meant was it allowed us to ship this brand new infrastructure and brand new architecture without having to change everything downstream. So, for us, that was a really great way to show wins inside the business and also move things forward while also making sure that everything that came before will still work. Like, if we didn't do this, we essentially would have had to update it, like, for client applications. And for those teams, they would have stopped development on the product for that long, just be waiting on that. And we'd be bottlenecked by deploying this whole new infrastructure by those services. And, you know, this is an absolutely throwaway part of the system. It effectively meant we were building two APIs. So, we built the brand new API and then this intermediary API as well. So, really, we added our development time by about 100%. But the way that we looked at resourcing, we felt like it was the best way to approach this problem. And we're looking to throw this away as well. All of those client teams have gotten their roadmap to be moving past the old API and the old schema and moving to our new API. So, I wanted to show you a little bit about how this works. So, I haven't started with the HTTP request here. But basically, what we do is we, there's a controller, it calls out to a service. And in that service, what we do is replicate what the old contract of data was. So, for this case, it means fetching an asset which for us is like a video. And then all of the comments associated with that asset as well. So, what we did is we wrote this really simple request library thing here that you can basically execute those in parallel and then joins them all together and then we can take that data and do something with it. And for us, what we do with it is actually encode it from a new type to the old type. So, we're going from a new database schema to an old database schema. And I'm a huge fan of protocols in Elixir to do things like this. It's a really good way to just get these kind of polymorphic generic kind of things inside of your code base. So, in our case here, we call an asset a file reference in the old world. And what we're doing is basically moving a bunch of attributes around and then casting it as a, as that struct for the old world, which we call a file reference. So, you can imagine the input here being a new video or a new asset and then the output is an old file reference. And then what we have to do is basically serialize everything. So, we made this really janky little typing code a thing that takes in a type, basically turns everything into a string. And I just did this to troll my CTO, so. But we take everything as a string, sorry, all the inputs and then turn it into a string and serialize those back out to the clients. And those clients are none the wiser. They think they're still interacting with that old system. Okay, so the next thing I wanted to walk through is a hotly debated subject. And again, I think Andrew did a really great job of speaking at length about how to structure umbrella apps well and thinking about context and breaking things down. So, we use this single mono repo, which I think a mono repo has been very effective for our team size. Everyone can contribute to this one single source of code. There's 11 apps inside of it, which is probably pushing it at this point. We're probably going to start to think about breaking those out at some point soon. But for now, it's been great. It means we're all working in one place. All of our deployments are kind of quite simple. Every time we push, we can deploy off of all of this. So, the apps look like this. On the left-hand side, we have four Phoenix apps. So, we have our API, the support tool, Munger, which is the intermediary API, and then this auth app.  So, the shared components on the right here are kind of interesting. Really, these could be extracted into a private HEX organization repo-y things now that that exists. But we've kept them inside of our umbrella application for the time being. But we have stuff like everything that we do is stats D. We want to share across all the services. We don't want to have to rewrite that every time in all the configuration. So, we just dump that in a little app and import that. Similarly, middleware. So, we have to have a lot of secure headers. Security is a big concern of ours, working in the video space with large clients. So, we put all of those kind of shared components in this middleware library. And then the DB is basically configuration and access into the database with some shared kind of types as well. Things like that that we need to make use of elsewhere. So, just highlighting a few of these applications. So, all of the five here that I've highlighted are actually things that we deploy and run. So, these are apps that aren't just shared code. They're actually things that we look at, run, and people can make requests to, and things like that. So, all of those applications are built and deployed as separate Docker containers. We use CircleCI 2.0 extensively to do all of this via distillery. If you're not familiar with CircleCI 2.0, definitely have a look if you're doing Docker base workloads. What we've been able to do is basically paralyze all of our builds. So, every single build there of all of those applications, each one, it will be the slowest build is the one that will take longest. So, for us, they all take about five minutes to run. We do things like blue, green, deployers via ECS. You kind of like just get all of this stuff out of the box with ECS, which is the elastic container service. I should have said that earlier. And everything we do now is auto-scaled. And we use CPU and memory threshold alarms to do that. Works really well. We basically set it and forget it. And we can see some nice graphs about where we've auto-scaled and how we've met the traffic demands as well. The other part of the app I wanted to walk through was the core application. So, for us, core is, as it might sound, it's all of the core business logic. So, it's everything to do with services, everything to do with persistence, access policies, lots of deferred logic, everything like that. And what we've done is separated it out into two contexts. One for accounts, so everything to do with managing teams and users and things like that on the platform and everything to do with projects, which is everything to do with, like, assets and adding people as collaborators and things like that. We have lots of different kind of collaboration concepts in the application. And effectively, we make use of the core business logic layer inside of the API and the support tool.  And actually, the fallback controller in latest Phoenix, as well, has been really great because you basically just, like, write a two-line with statement, and then you handle the error case just magically in the fallback controller. It's well worth looking at. And inside of this core layer, we have a lot of tests, so lots and lots of tests. They run in about 10 seconds. They're very database dependent, which is usually the bottleneck for us. But it feels good to have that many tests coming from zero. It definitely is, like, a win for us. So the third thing I wanted to walk through is our event system. So this might seem really familiar. Actually, it's really interesting how well some of these talks have flowed together today. So good speaker selection, everyone. But our event system looks a lot like what Andrew was talking about previously. So we have this single kind of system where every single change in the app is pushed through this event bus and broadcast to lots of different consumers of that change. So it allows us to have this really powerful way to decouple parts of the system. So we use it for things like audit logs. So you want to say what person did what thing in the system or analytics. I'm sure everyone here is making use of something like segment or something like that where you want to say this event happened for this user. So we use our event bus to decouple these concepts to keep our services really clean. And they just push these events. And everything internally is implemented using gen stage and protocols. I'm not going to show you the gen stage code because it's literally taken from the example on the gen stage documentation for how to do the gen event replacement. And it's been working really well for us. So just to give you a little bit of an example about how this works. So we have our service here where in this instance we're creating an asset in the system. We have some function that does the create. It will return an asset back with the nice tag tuples that we use in elixir. And then basically we create a struct. So we have every single event in the system has a corresponding struct with it that acts as that kind of place holder so we can hook in and do some nice protocol magic in a minute. And then we push it through our broadcaster. So that sends it out into the event bus. And then as I said I'm not going to show you the broadcaster here but this is one of the consumers. So our auditor here acts as a place where we write audit logs in the system. So an audit log is what person did what thing on what resource. And we write all of that through this kind of this model. So what we'll do here the event will come into the consumer. We'll basically call the protocol with the event. And then if it if it returns something that we're expecting. So if the with statement here matches then we'll actually insert some kind of audit entry. It's very simple. But it works very well for us right now. And we've scaled this pattern out a lot. So this is what the implementation of that might look like. So we say on the right hand side here you've got the asset created event. So we are implementing the type of protocol for that event struct. And then basically that thing will return an audit. Which if you look at the previous slide here you can see that it was expecting an audit and the with statement. And we just build up that struct return it and then we're done. And what we've done is actually scale this pattern out for a ton of consumers. So we have audits we have analytics we have all of our notifications. We have things that broadcast to our socket service. Lots and lots of different events and they all throw flow through this singular event bus and then get sent out accordingly. So at the moment just a caveat with this in our implementation. So the broadcaster here is going to notify every single one of those consumers. Because that's how we are using gen stage right now. There are definitely different ways to do this as Andrew showed you before. Where you might actually have more of a pub sub model. Where you're saying I want to subscribe to this type of event only. And then you maintain some kind of list of the pub sub. And all of our consumers here are implemented as dynamic supervisors. So we get lots and lots of processes processing each one of these. Sorry requests there as well. Cool. So the last thing I wanted to talk about giving you the overview of the system is how we went and moved these millions of records that we have. So how we went from DynamoDB to Postgres. So what we made use of here is flow. I'm not sure if everyone is familiar in the audience with flow. Flow is an abstraction on top of gen stage. It kind of acts like a stream where you can take in a source of data and then fan out and have loads of processes basically act on that stream of data. So it's a really great use case in our case for we basically stream the entirety of a DynamoDB table and then fan out with loads and loads of processes taking those DynamoDB records, translating them into Postgres and then inserting them into the Postgres database. So flow is a great tool for data processing and I'll show you a little snippet of flow code and how easy it can be to use something like this in your own code base as well. But our largest table was nine million records and in total to sync every single table so there was about 22 tables in the system. It took us about an hour of downtime so we had to turn off the old database, shut down all the system, basically turn on all these data migration tasks, wait an hour for everything to finish and then turn on the new system. Yeah, that don't do that, don't ever do that. It was a good idea but it kind of worked for us. So how we implemented this was we basically have all these schemers that are really thin wrappers around DynamoDB tables. So you might actually recognize this because it looks a lot like Ecto. So we wrote a really simple wrapper that acts like an Ecto kind of schema so we can define lots of different types here. So and this thing will do all of our type coercion from strings into the appropriate types for us as well which made it a bit easier to deal with some of this data.  The statics.measure part is for us is how we we get statistics and stats off of this kind of function call and then inside of that what we're doing is we're saying scan from the Dynamo table so a scan will basically say give us lots and lots of information and we do that per we do a limit here of a certain amount so we'll put each table differed on the amount we needed to scan at a time depending on the size of the data set. So we basically start the stream off by scanning the table and then we start the flow process here. So flow has a single from enumerable which is from as you might imagine from a list or something and then that turns it into a stream and then we're mapping over that turning each one of those things that's coming from DynamoDB into us into one of those Dynamo structs that I showed you before and then we basically partition so what we're saying is for all of those things we partition by the IDs and then we fan out to lots and lots of processes and then each one of those processes in turn is then calling a function that's taking that Dynamo schema and turning it into a Postgres schema and inserting it. I know that's a lot so I can ask answer questions at the end and honestly I wish I had more time otherwise I could go through lots and lots more of this but if you want to speak about this grab me after the talk and then what that from record looks like so this is the function that took the Dynamo schema and then turns it into a Postgres schema and inserts it into Postgres so we're basically doing lots and lots of those in parallel as many as we can do and we spent a really long time kind of tune in the workload so we could do this really quickly and efficiently so lots and lots of like per table tweaks and configuration needed to do this kind of thing but every single thing that we did here we ran in its own docker container so all of these table processes are running in parallel at once so we're parallel processing and then parallel inserting all these different records and we basically just got like the biggest Postgres box we could we could get and just let it go wild and yeah and that's how we did like I think it was like 20 million records and that's how we did that in an hour and 10 minutes in total so very very cool elixir has many tricks up its sleeve so flow is part of the gen stage the library I believe but I could be wrong is that no it's separate thanks James he gave me a nod so it's really worth looking at if you're doing kind like these large data processing workloads and you don't need something more like a Hadoop or you know some of those bigger map reduce kind of jobs because flow is effectively doing a lot of that for you so everything here we kind of we kept running this thing weekly kept dumping all this data we were doing trial runs all the time to try and make this process smooth because you know taking all of your customers data and moving it to a new database is kind of terrifying so we did as much as we can that could to mitigate the the risk of error here as well cool so I wanted to talk a little bit about some of the challenges and some of the takeaways that we've had from doing this so just a I mean there was a lot to go into and I know I've gone through some of this quite quickly but I'll just recap some of the biggest problems that we had during this journey so as I said before when you have a system with no tests where basically everything is hearsay about how it works re-implementing that system is really difficult and error prone so we ended up with tons of bugs tons like it especially in that might that intermediary API because oops we found out that someone actually implemented something a different way and the system should have worked in this like a way that we wouldn't have expected and you know there wasn't many ways to deal with that problem for us other than lots and lots of manual QA and that was a very error prone and slow process but we eventually got there ironed out most of the kinks it just it took us like another six weeks extra from when we thought we were going to ship this system to when we did but I think that's a reasonable amount of time given the complexity and what I've kind of gone through today as I said before we had lots of new developers trying to learn elixir and the and shipper project that's you know it's a lot to take on we these were brand new members to the team as well so immediately you don't have that kind of institutional knowledge about how things should work so what we really did is we had a few more experienced members of the team kind of going forward establishing a bunch of patterns laying out the foundations and then letting those other members kind of come in and work on top of that and that was actually really good that worked really well for us we implemented lots of style guides we had credo as a linter we're trying to enforce lots of quality in the code so everything has has to have like documentation and all of our public functions we generate docs off of everything here as well so we enforce lots and lots of conventions on the team I think that really helped people get over some of the learning hurdles as well and then the other thing here is just shipping a new system and a brand new database where you haven't run the workload at a production scale is a very difficult thing so especially in postgres where the performance between what you might be running in dev and production might be wholly different right so in dev you might have all your qa environment you might have a few thousand records and then in production you might have a few million records and the way indexes work and things like that are very different between those two so what we actually ended up doing is replaying a bunch of traffic and really thinking about those kinds of ideas before we ship this thing so we could get handle on how it would run in production and then the other part of that as I said before was we used a single service as a test case to get an elixir into the team and making them understand how to run it at scale and that was that was like insanely valuable so if you're thinking about doing a migration like this and you can't break it down like start with something small and then work inwards from there so a few takeaways so the first one was elixir definitely was a huge win for us but it might not be for you right rewriting a system just because there's a new language might not be the best idea right think really carefully before you embark on a rewrite really think about like what are you trying to solve are there other ways to do it like you know that second system syndrome is a thing like you're doing it is going to take you twice as long cost you twice as much so just think very very carefully about doing it elixir was great for us because of the concurrency model basically and the fact that we could ramp a lot of our developers up and use lots of these really great attributes of the language to kind of promote lots of the explicitness and and get a much more maintainable code base out of it so but you know I think your mileage may vary is a saying that Desmond and I say a lot on the podcast and actually is definitely true here as well takeaway to is basically if you do a rewrite don't move databases at the same time like yeah this is basically like flying by the seat of your pants kind of stuff you know moving millions of records and the business critical amount of data is is scary it's terrifying and I would keep reassuring the CEO like yeah I'll be alright well yeah don't worry we've tested it we tested it but even on the day when we're doing it I was kind of crossing my fingers and hoping for the best right and we we actually developed a lot of tooling to help us with this process so we could easily like get a single customer's data if it didn't migrate properly and things like that and fortunately we only lost one customer's data during this entire migration but we were able to recover that data as well so we lost none so that was a win and the third takeaway this is I think I put this in like every talk is like absolutely love this article so this quote comes from an article called write code that's easy to delete and not maintain and I really really love this quote from Tef which is that good code isn't about getting it right the first time good code is just legacy code that doesn't get in the way and that's how I like to think about our systems like we've re-architected it and we've embarked on this new journey and hopefully we've put in a lot better kind of boundaries around different parts of the system so that those bits of code doesn't get in the way and if we do need to get rid of it it's much easier to do so right design your systems like that think about think about keeping things nice and isolated my module is basically just a grab bag of like related functions right so it gives you a really easy way to like collect all these things together that do the same thing and start thinking about that modularity before you embark on doing a project like this or any kind of big system in general cool so thank you very much and I'll definitely answer questions so there's a question oh yes so he said how can you lose one customer's data that's a very good question and I don't know the exact answer but we basically had a situation where we had we had a bunch of duplicate users in the system so and we basically when we're doing our import we use the user as like the thing that we found out and down from so if we didn't have them there and we were trying to cross-reference the records and stuff to get guarantee the consistency and make sure it's there but I think that user escaped our wide net yeah so the question is did you keep the DynamoDB database around before and after migration the answer is it's still there and we just like lowered the read and write capacity to one but we basically have a backup of that data which is good just in case someone comes around and says hey I don't have any of my data and I haven't logged in for six months and we could hopefully help them yeah I'm still nervous to delete that to be honest I think I always will be but and I think what we're going to do is actually take a backup and then have it in S3 or something yeah great question so did I have a contingency plan to go back if things didn't go right absolutely so we had clear so we actually ran like drills of doing this deployment about three or four times before we did it and we had clear rollback points at every step on the way until we did the data migration like when we turned off all the services was if we would have turned everything back on and then been like oh we needed to revert back we would have had a gap in data and that might have been an acceptable loss for us given the severity of the bug like fortunately that didn't have to happen but we practice that and our drills as well so that was a question about flow and how did we get to our ideal batch size using flow the answer is basically a lot of trial and error different for us that lots of the sizes of the tables was were different the records inside of the table so I mentioned in that one of the slides that one of the largest ones that we had was about 100 kilobytes per record dynamo actually has a scan limit of how much you can return in a single request so we were bounded by that and then basically it's like how much can you pump through dynamo and then fan out and really that was just a massive like trial and error tweak it see if it took longer run it again drop the DB run it again it was yeah lots and lots of trial and error but that's why we started doing these like migrations like way before we got to the the finish line you know I would say we would I'm sorry so the the follow-up question is how many weeks did it take I would say we were doing that for for about three months probably doing it once a week and for us we needed all the data in dev and our like QA dev environment because that's where we're doing all of our acceptance testing against the new API and the intermediary API so we needed to run it all the time anyway great question as well so the question is did we can continue feature development during this time the answer is somewhat so unfortunately again being in that startup environment you know yeah it's very hard to stand still we did pretty much like get to a point where we'd say we're not adding any new features to this API but the front ends kept moving but then what we would try and do is stay in sync between the client teams and the back-end team to make sure that you know their changes were reflected in the new schemers and we weren't going out of sync there but yeah oh yeah great question as well and how do we determine the percentage of test coverage I think like percentage of test coverage is a bit BS to be honest but and I think a lot of people disagree with this but for me like anything over like 70% seems okay what we really optimized for was testing the business critical paths and lots of the kind of common cases I mean the coverage there is like there's so much right like it's a whole three-year-old system re-implemented over again so I'm not a look I'm not like dogmatic about we must have a hundred percent coverage or anything like that in my team I just think every feature has to have tests and we need to make sure we're testing the main cases and hopefully we can use something like property testing in the future to try and improve that as well and yes the question is we had three developers who are learning elixir and I mentioned that we had some patterns in place and what were the examples of those so and one example of that is I actually did all the bootstrapping of the app so I would go in do the controllers break out the service break out the policies so the things can someone access this lay out all of those foundations and then get it code reviewed and walk through and do a pairing exercise with one of our other developers make sure they are fully up to speed with that and then they could run and follow a lot of those patterns but there was a lot of hand-holding in the beginning but now they're you know they this team is like running off and building new features and code in themselves and everything's like great so and we've actually brought on more members of the team now as well and they're able to teach the other people so it's been great yeah and the last question so the question is did we consider any other framework beside Phoenix so do you mean outside of elixir as well or right and we didn't I I'm actually of the opinion that Phoenix is small enough that you know I don't feel like there's a lot of bloating there like the whole like Sinatra rails thing like I understand because Sinatra felt a lot smaller than rails did and there's a lot of superfluous stuff in rails especially now with like action cable and stuff but I think Phoenix is small enough that if we didn't need some of it like we just remove it so like channels in the API we didn't need so just get rid of it but yeah I'm actually not that familiar with any of the other web frameworks either I guess we could just use plug as well but I don't know if I'd want to do that cool thank you thanks a lot Chris


Hello, hello, it's Julie. Welcome to PD and your PJs. I'm so excited to be here with you tonight to talk about really easy activities you can try with your grade six through 12 students. I'm just honored you would take time out of your busy work week to join me to talk about CISA and I have some really great ideas and tips for you. So I can't wait to connect with you in this webinar and at the end when we have time for questions. Like I said, my name is Julie. I was a high school ELA teacher for 18 years. I was using CISA every day those last few years with 11th and 12th graders. And I'm now on the teacher community team at CISA. I work with Angela if you've ever heard her present in any of the other webinars. I'm on the community team with her. You can find me on Twitter at EdTechJulieJay and I would love for you to give me a follow if you happen to have it open on a device. I share lots of resources and tips related to CISA. And of course you can connect with our whole team on Twitter at CISA. Now I do want to tell you that we're just talking about the first few activities that your students might want to do in CISA. If you're just getting started, this particular training is not going to go over how to create a class or how to add students. So if you're in need of that kind of information, definitely take a look at our other PD list. You can find it at web.cisa.me. Or you'll get these slides here in a little bit. And when you get them, just open this one and click on that link there. You want to look for some webinars that have the phrase brand new in the title. So we have brand new part one and brand new part two and look for the ones for six through 12th grades. These slides are coming to you along with the recording. So you'll get those in just a little bit. I'm going to pause really quickly and look in the question box just to make sure nobody's having any trouble connecting. Okay, it's looking good. You can type in any of your questions related to these activities in the question box and I will answer those as we get towards the end of our presentation tonight. Okay, so when we're talking about activities and the really robust activity library that Cisa has for you, it's really a twofold purpose. One, we want to make sure you have lots of ideas as far as what your students can be doing with Cisa.  But beyond that, we want you to know that if you're using this activities flow in Cisa, you're actually going to save a lot of time. You're going to be able to collect your student responses digitally all on one screen. And you'll have to lug home fewer folders and binders and other papers to grade. Or even if you're used to collecting your students work digitally, instead of having to click through 20 different links or files. In this case, you'll be able to see student responses all from one screen. And I can't wait to show you that in just a few minutes. So we think not only will this give you great ideas for how to get started with Cisa in your classes, but it'll also save you some time in the long run in such an easy and quick way to see what your students know. So I'm going to show you a couple of screen grabs of that and even show you that in a live demo here in just a minute. So when you're in a Cisa class, if you're just getting started and you have created a class, but maybe the class doesn't have any student posts yet, you might see a screen something like this. You'll see this blue button encouraging you to browse the activity library. Even as you're just getting started with Cisa, you can click that blue button when you see it in Cisa. And Cisa is also going to give you a few suggestions that are just right for your student's age. So if your class is set at eighth grade or ninth grade, for example, Cisa will give you some appropriate suggestions there. And you can click on any of those to go right to those activities in the library. But I'm also going to show you some other ways to get to the library as well. And the focus of this particular session is to talk to you about the getting started collections for your grade. And to be honest, as someone who was using Cisa every day with older students, there are just some really wonderful activities waiting there for you, whether you teach sixth grade or twelfth grade or any in between. And they're also sorted kind of nicely or organized kind of nicely by subject. So you can look for the subject that you're teaching and find something that's going to be just right for your students. Here's the thing I was talking about too, and I'll demo this for you live in a minute, but it's really handy that you're going to be able to see your students' responses to these activities all in one place. So this is just a screenshot, but what you can see here is that a teacher has shared this activity with students, and then quickly, just in one screen, the teacher can see all of these responses. If this is an activity that you think might interest you, you can click on the photo here later when you get the slides, and it'll open that link for you. And here's something I was using Cisa for every day, just kind of a ticket out the door, a quick formative assessment and exit ticket, and your students can do that really quickly, like creating a note in Cisa. And look how handy that is. You can see which students you're waiting on, and you can see everybody's response all from one screen. So I'll show you that live in a second, but I just wanted you to see it on a screenshot because it's so exciting. Okay, I am going to hop into the library live with you. I included this particular screenshot here just to remind myself to say to you that we have a lot of subjects represented in our library, not just what you would think of maybe as some of those core subjects like English and math, but we also have a lot of things for your specialists.  So don't forget you can search for those subjects as well. Okay, I'm going to pop out of my slide and go right into a Cisa demo class. Now you can probably tell that we use this type of thing in a lot of our trainings and PDs workshops, things like this. So this is a demo class. These are pretend students, but we've populated it with some samples of student work so that you can see what it looks like. You know that right now I'm in the journal view of Cisa. We're talking in this particular training about the activities tab and what it will be like to share activities with students. This is what it looks like when I see a list of all of the activities I've ever shared with students in this class, including the one you saw a screenshot of a minute ago, the exit ticket activity, or even sketch your goals, which I showed you on a slide a minute ago. So that's kind of what we're talking about is the activities. Now I showed you the blue button that might be waiting for you when you first create your Cisa class, but this green button, this green add button is probably where you're always going to go to get to the library. You can do a lot of things from this green button, but today we're talking about the library, the activity library. So you're going to click that green button and go right to the library. And remember, my class that I'm demoing from right now is an eighth grade class. So I'm going to see that Cisa is already taking me to activities that are just for eighth grade. But don't worry, you can always filter by grade. So I can switch to another grade. And I can filter by subject. So I can take a look at all of the subjects available to me. Or I can even type in a keyword and type that right there and see what might come up that's related to that word or topic. So I have lots of different ways I can find activities. But I'm really excited to show you these getting started collections. Okay, you want to make sure that you're using the scroll bar as needed, because like I said, we have some of those core subjects represented here, ELA, math, science and history. And there's five activities in each of those collections. But the more you scroll, you'll also see we have some arts and music themed activities.  So you could use that in any subject. So hopefully you can find a collection that works for you, something that you're teaching. I'm going to demo right now with ELA, but there are great activities in all of these subject areas. Okay, so when I click on that, this is the collection. And just quick reminder, the collections at any grade level are represented by those colorful rectangles. These are collections, colorful rectangles. But again, we're in big kid grades here. So I'm going to remind you what that looks like. You have the scrolling bar when you're in an older class. Okay, so I'm going to demo from ELA. And then my collection is going to have five activities. Now, the interesting thing about this is that in the getting started collections, which are designed for new users of Cisa or students who are just getting started with Cisa, all of these basically introduce one of Cisa's creative tools at a time. Okay, so you're really just learning the basics of Cisa as you complete these activities. So you can just see at a quick glance here that this one is focusing on a student learning how to take a picture in Cisa. And remember, you can do that across any device or platform. So if your students are Chromebook users, that's fine. You can still take pictures on Chromebooks, take a picture, label it. That's what they're learning in this particular activity. They can also record themselves in a short video as they respond to a question that you've given them. So they're using and learning the video tool here. Here's that note tool. We looked at it a minute ago with the exit ticket activity. So maybe they can just do a quick journal prompt answer or they could maybe do a quick formative assessment by typing a note. This is the drawing screen. So you can see they're kind of using that whiteboard and here they're actually reading something that you as a teacher can share as a link. And there's a sample here for you that we've given you in this activity collection. Okay, so these are the five getting started activities, but I know what you want to see is how you would actually get those to your students. So we know that we can browse the library by grade or subject or keyword. But once I found an activity, maybe even one of these from the getting started collection, how would I get it to my students? Well, I'm going to tell you the key here is anytime you find something that you want to use with students, you want to go ahead and give it a like. You can see I'm kind of toggling my mouse around it so you can see that it's turning pink and white.  So I'm going to like it, give it a pink heart. And now I actually have some options available to me to either organize it in the collection or to kind of customize it for my own students by editing it slightly. And I have those options available to me now that I have given it a heart. But I can just share it right away with my students without doing any of those other things. So I'm going to do it from that big green button before I model that for you. I'm going to show you a couple other features that you can see here on my screen. These are teacher notes that explain the activity. If you are new to see saw and just getting started, please read these notes because we're giving you some information and context about what your students will be learning by doing that activity. You can also see that these little seesaw icons appear in my instructions here and you might be wondering how did they get there. I'll tell you that they're from these icon shortcuts and I can share a link with that link with you about how to get to this shortcut page. I can put that in the slides for you to you don't have to memorize these or anything. This is just kind of fun for you to add these images to your instructions. It's really important for emerging and pre-readers. So teachers who teach younger grades are really excited about using these icons. It might be less important for those of us who are teaching older students, but it still does make them look pretty cool. That's how you get those icons. So I'll make sure to share a link. And then what I was saying is that you want to click this share button to get it to your students. Now if you're teaching big kids, you might have a different class every hour like a first hour ELA second hour. So you might have a long list of classes. I know I did when I was in the classroom. I have a lot of demo classes right now. I'm only going to select one for now, but I could select two at a time if I wanted. If I was teaching first hour 9th grade English and second hour 9th grade English, I could select them both, you know, to get the activity to both of them. This is how I'm getting it to my students. You can also see that I have the option here to share this activity with grown-ups. If I want to share it with somebody else in my school, I can share it on social media, share the link. You'll even if you're using free seesaw, see a tab here that says email to teachers. So you can click this to get it to other teachers over email. Super handy. I'm going to share it with my class now and then seesaw is going to remind me that my students just need to sign in and respond. And if I want to see if that activity made it to my demo class, I can go right there and you can see if you're watching me in real time. This is exactly the date and time that we're at right now. So you can see that I shared it to my students. Of course, I don't have any responses because this nobody's in this class live with me, but you can see that I shared it with my students and it's right there waiting for them. So this is my journal view like my feed of anything students have added to seesaw. So nothing from that activity is in the journal yet because they haven't submitted any responses. But once they would do that and I would approve it, then it would go permanently in their journal. Okay, so what's nice as a teacher is from the activity tab, you can always click this gray bar to see how many responses you have, how many you're waiting for. And it's this screen where you end up seeing all the responses in one place like we've talked about a few times tonight. Okay, I'm going to go back to my slides and then I'll take a few minutes to answer questions. If you have any excuse me, remember the best place to reach out for help if you're stuck with activities or anything else as you're getting started with CISA is our help center. It's help.cisa.me. You have three different places you can get help, you can ask a question here, you can type something in the search bar or you can even interact with our CISA assistant. So several different ways you can get help from the help center. As we conclude tonight, I just want to give you a reminder to you that a survey is going to pop up on your screen as I conclude or after I conclude and push the button to stop the webinar. Please take just 30 seconds or a minute to give me some feedback and answer the questions. We use that feedback to help make our sessions better. So I appreciate your time and any honest feedback you can give me about our presentation tonight. Of course, if you can interact with us on Twitter, Instagram or Facebook, we do have grade level specific Facebook groups. So I really recommend you getting into the middle school or high school one, depending on your grade level. It's a great place to connect with other teachers using CISA. And then of course you know about PD because you're in PD right now, but don't forget to keep checking that we're going to be updating our schedule soon for September. And then you can watch any of our recordings that are still on YouTube. I'm excited to get in the question box and see what you might have waiting for me. I think what I'm going to do is stop the recording tonight, but then stay online for a couple more minutes. So start typing in those questions and I'm super happy to help. Thanks so much for joining me tonight. I hope to see you again at PD really soon. Bye bye.


Okay, so let's keep moving forward and let's focus on actually Cleaning up the branches a little bit more So what I want to do is I want to actually add a little bit of droop to the branches and and as they Start to increase in their y direction.  All right So let's take a look at how we are going to do that Okay, so the first thing I want to do is I kind of want to rearrange how I'm doing This particular technique right here because I need access to each individual copy And I want to work with each individual copy to do that drooping effect. Okay, so to do this I'm going to drop down a for each point Loop right here. Okay, and I'm going to wire in the Result of this branch normal directions right here Okay, these are all the points that we have right there and you'll notice now what we can do is I can come into this For each end and turn on the single pass and now I can work with each individual point and I can get data From it specifically this y pause data now If you're familiar with the copy to points node, you can also copy the point attributes So let's take a look really quick at these two different techniques here. So what I'm going to do is I'm going to wire in The result of this for each begin and then wire out the result of copy to points into the for each end right there okay, and You'll notice now. We can work with each individual Branch and start to do operations on that. So we're going to perform that drooping amount right here. Okay So let's keep moving forward. Oh, I want to take the result of the for each end and actually wire that into the merge there All right, so that way we have access to that. Okay, so Let's take a look at at least accessing the point So like I was saying before inside of the copy to points node if you have this copy to points or copy points Attributes right here turned on you'll notice that in the final copy that it is actually copying over That y-pause value to each point on here. So you'll notice that if I were to change the amount of points I come back to this node We now just have three values because there's three points here. Okay So There's a couple ways you can do this and the reason why I'm talking about is because I want to show you the two ways That's one way right is you can just utilize the value coming in The other way to do it and the way that I'm going to use here I'm going to drop down an attribute wrangle node. Okay So I'm going to do some more vexing here. All right And I'm going to pipe in the result. This is the individual point that we're currently working with. Okay All right, and it's always point zero right because we're working through each of the points So it's always going to have an ID of zero, which is awesome. We can always count on that All right, so I'm going to wire that into the second input here and remember that the inputs are zero base So it's zero one two three Okay, so inside of this particular node here What I'm going to do Actually, first let's go in here. Sorry about that I'm going to drop an attribute delete just to prove that this is in fact working So I'm going to say attribute delete and we're going to delete that y pause All right, so now we're not getting it copied over All right, so we have a clean value and we can also change this gradient value as well. So Let's get rid of that for now There we go. So now we have nice clean attributes to work with All right So in our inside of our attribute wrangle node here, so let me wire this up like so What I want to do is I want to capture the current points gradient and its y pause So this is just the other technique. Okay So what I'm going to do is drop down a comment and we'll say get y pause and gradient from current point All right, so in order to do this we're going to utilize an expression or a method called Point all right, so I'm going to first do f at because we're going to store this in a value. Okay, so we're going to store it as y pause So that's the name of the variable is equal to point. That's the function that we're going to use And now the point takes three Arguments here. So let's go into the help here Really quick. Let me launch the help It always helps to bring this up during the videos just so We can all see So there we go go to the point of x function here Okay, and you can see that it takes the geometry we want to work with The name of the attribute and the point number that we want to work with Okay, so In this case, we want the geometry from input one right the second input So we need to put in one for that and then the attribute that we are looking for is the y pause And then the point that we want to get it from is point zero because that's the only point that we have All right, so point zero is that only point over there Remember if we come up here to that for each begin, that's the only point we have so we want point zero Okay, great. So let's do the same for our gradient. So f at gradient is equal to point And we're going to get it from one and the name is gradient And we want it from point zero and we should probably name this something else because i'm going to use that same So we'll call this other gradient or we'll call it the main gradient We'll use this value for controlling the global drooping amount All right, so now you can see we basically get the same result. All right, we're getting those values So we have the main gradient and y pause All right, and we can actually change this y pause just to make this even more Evident we'll call this the droop direction like so And there we go. So we have droop direction the main gradient Cool, and that value has been put on each individual point there Okay, so with that what I want to do is I want to put in the gradient for the branch All right, so the main gradient is the gradient for where the branches are being placed on the trunk All right, so that's going to be useful information because then we can change the value as they're placed The gradient that we're going to create in here is the gradient for this particular branch And that'll control how it droops over the length of the The branch because at the the root of the branch, we don't really want it to droop that much It needs to kind of fall off Okay So we need a gradient in here. So I am going to say float gradient this time. I'm not going to store it into an attribute All right, I'm just going to put it into a variable And that gradient again is at pt num divided by at num pt minus one All right, we'll put that inside of parentheses like so And then we'll do the cast Like that There we go Okay, so now we have a gradient on this particular Branch line right there. Okay. So what we want to do first Let's go and just droop everything By its droop direction amount. So we'll do this kind of one step at a time So at p dot y right because we just want to droop it up and down. Okay, so at p dot y plus equals All right, so if it's negative, it will be a negative droop if it's positive. It'll be a positive droop. Okay Equals our droop direction. So we're saying at droop direction like that cool And you can see that it's it moved the whole thing. So if we come back up here and template it It moved the whole thing down. That's obviously not what we want So what we want to do is we want to then multiply that by the gradient that we created so the branch gradient right here So we're going to say ch ramp because we want control over it, right? We don't just want a linear droop and we'll call this the droop amount And we will pump in the gradient here. All right, and then we'll make that particular curve Okay, so now We should be seeing The droop amounts in here. Oh and the reason why that's not working is because I didn't make it a attribute I just made it a local variable. So there we go Perfect All right, so again, we can move this different sides here And we can droop this however we want now All right, so The best thing to do here is to just make all these guys I'm holding down shift to select all the points there and just set them all to a piece of point And that'll make it nice nice and curved Now you'll notice that when it droops The lines get a little bit longer or the segments in between each points gets a little longer Now there is a fix for that But it's going to be a little bit too complex at this phase in the series So I'm just going to leave it at that for now. All right We actually want to be able to rotate each segment and keep the segment length I'll do another tutorial on that here pretty soon, but I want to still keep this relatively basic Okay, this will suffice for now. Okay. So with that, let's take a look at the final result here So let's just turn off that single pass in the for each end node And let's take a look at what we get All right, so now we're starting to get something. That's way more natural All right, so if we take a look at the final results, we now have A pretty natural drooping amount But what I want to do let me actually Pull up where the branches start here a little bit Cool. So what I want to do is I actually want control over the global droop Okay So in order to do that I am going to come back into my Attribute wrangle, which we should name. So we're going to call this the branch droop like so All right, and we're going to create another Variable so I'm going to put these or this line into A parentheses just to kind of section it off there And we're going to multiply this by a the global or the main gradient here So I'm going to create another ramp and this is going to be a ch ramp and we'll call this the global droop And this will just allow us to control the droop over the whole length of the the trunk here Okay, so I'm going to pump in that main gradient like so All right And now you can see that they all aren't drooping anymore. All right And let me hit the create spare parameters and now you can see that Because we have this linear curve here at the bottom. We're we're not really getting any droop and at the top They're starting to fan back up. All right. That's pretty cool actually in and of itself And what we can do now is we can put in a point in the middle here All right, and then increase the bottom to get just a little bit of droop at the bottom there All right, and we can just again overpower all this stuff. So now we have lots of control over how this looks Very cool and we can bring down, you know, the main droop at the top there So now you have lots and lots of control Cool. That's kind of what I'm looking for All right All right, so what we want to do now or at least what I want to do now is I also want to put in the p-scale and I know we removed it beforehand, but let's put in the p-scale here So I'll say at p-scale Is equal to our at gradient All right, so this is the global trunk gradient there And there you go. So now we have that control and again, let's put that into A ramp so we can control it All right, so we use that a lot I'm going to call this the branch size Or global branch size How about that? Make sure we put a comma in there And just close it out. All right create that spare parameter And there we go. So now we can have control Over how big the ones at the top are Maybe they get bigger in the middle And they're the biggest at the end there like so and again if you want to go and Smooth all that out. You can just select all those points by holding down shift And set it to b-spline And then if you want to control the branch size even long more we can just do something like that That's more of a global control as well All right, so that is the drooping there. So that's all I really wanted to do there So I think in the next video we will start to put in the smaller kind of fronds The branches that branch off the branch All right, so let's keep going. Thanks so much